I'm personally working on something like this for the ESP32, but written on top of micropython [1]. A few things are written in C such as the display driver, but otherwise most things are in micropython. We chose the T-Watch 2020 V3 microphone variant as the platform [2].
Our objective is to build a modern PDA device via a mostly stand-alone watch that can be synced across devices (initially the Linux desktop). We want to achieve tasks that you might typically do on your desktop, focussed towards productivity.
We did consider a custom OS, but decided against it for a few reasons:
1. Allowing somebody else to handle basic OS stuff allows us to concentrate on what really matters, the higher level stuff on top.
2. Having multiple threads in micropython is super simple and we are able to run many active apps at the same time, rather than having to kill them off [3]. Our background apps can continuously interact with the network in the background.
3. Code written for micropython can be easily run on other Python-capable devices.
The thing with threading in micropython is that you'll have to either rely on task priorities and cooperative yielding, or GIL, and both of them can be easy to shoot yourself in a foot with.
The CCCamp23's flow3rbadge also used micropython to implement its app framework st3m: https://flow3r.garden/
bArray 24 days ago [-]
We only have a few privileged tasks (scheduler, hardware, visible app). We ask that apps finish their processing within a certain time, otherwise we kill them [1]. Ideally we would have the ability to pause and restore them at will.
Our system is not all dis-similar from flow3r it seems [2].
This kind of approach feels like the modern version of BASIC + Assembly from 8 and 16 bit days.
bArray 24 days ago [-]
Exactly, but with more processing power than the BASIC + Assembly days, and more connectivity.
pjmlp 24 days ago [-]
Definitely, I always compare the ESP32 to a modern version of the Amstrad PC 1215, and we could already do quite a lot with it.
Hence why unless we're doing some kind of PIC like development, it is about time to embrace more modern tooling. :)
rvense 24 days ago [-]
I always thought Micropython deserved a C64-like computer (with some pins exposed on top).
d_tr 23 days ago [-]
I'd hate to be that poor little bastard, having to run 200 bullshit instructions for each useful one.
I'd probably just output a middle finger somewhere and halt forever.
mitjam 24 days ago [-]
This is really interesting.
Do you think the hardware would be a suitable platform for voice assistant type applications, with AI on server side, of course?
bArray 24 days ago [-]
This is exactly what we are investigating, to record audio locally (minimal processing) and process it off-device. I think it's definitely possible.
mitjam 24 days ago [-]
Ok, I'll give it a try. Maybe I can get my daughter interested in programming with MicroPython on such a watch, as well.
teamonkey 24 days ago [-]
You might be interested in the M5Stack Atom Echo. I believe you can flash them to work with Home Assistant (you could also just use the new HA Voice hardware).
Wow, even with local wake word. Looks like a nice retrofit for my Logitech Smart Radio.
jononor 23 days ago [-]
Is the project available anywhere? I have that hardware, and am on the lookout for demo platforms for the ML/DSP library I am developing (https://github.com/emlearn/emlearn-micropython).
anigbrowl 23 days ago [-]
That's very interesting. I'd been thinking about getting a T-watch. Do you plan to have it work with the S3 series as well?
nine_k 24 days ago [-]
If your apps run continuously, how's the battery life?
If you freeze them to save the battery, how do you handle unfreezing?
bArray 24 days ago [-]
We're still working out the exact process, but apps return a dictionary when they are put to 'sleep' to allow them to return to a previous state. The app state is stored in RAM currently, but could also be saved to disk. They can request that certain hardware is available for when they are woken, and they can request to be woken up at a certain frequency.
We can for example put the ESP32 into a light sleep for some time [1] and keep networking alive if apps require it. The idea is to just stretch the battery to the length of a day, shutting down the display gets a lot of the way already.
Meshtastic is another project that has recently made serious strides[0] in their UX on the Lilygo T-deck (and similar ESP32 devices), but specifically regarding LoRA-enabled devices.
It's still on a branch, but I compiled and ran it, and now I have two T-decks that can communicate with eachother off-the-grid without a smartphone attached to send messages; it's actually usable in emergencies now, which is why I bought the devices in the first place.
Currently in the process to deploy a mesh from me to my parents and family.
What's the data transfer speeds you can attain with LoRA devices like these? My understanding is that they're geared more towards bytes per hour than bytes per second, and other than some kind of "I'm alive" message, I don't get the impression these devices are very usable for communication.
gh02t 23 days ago [-]
Meshtastic let's you select different compromises between throughput and range. The main limitation is that everybody has to agree on the same setting up front to be able to communicate with each other within a network.
The default (and most widely used, including on the de facto public channel) setting is one of the longer range options and gives you around 1 kbps, and it's quite usable on the default for normal texting and positioning telemetry with a handful of active users in my experience. The fastest setting is over 20 kbps and still gets ok range, the slowest is a mere 0.09 kbps and is only useful in some limited scenarios.
It's legitimately useful, and I actually use it regularly. Mostly for hiking, as I do a lot of hikes where we split and meet up. It's also far cheaper than commercial options, and getting close to being as polished as the commercial options like Beartooth or GoTenna.
It’s fine for text messages. It takes 5-10 seconds to send a kb of data (1000 characters roughly) at its longest range setting. At shorter ranges it can do upwards of 20k per second.
numpad0 24 days ago [-]
This appear to be a window system and desktop environment than OS, but isn't it that ESP32 user code always runs atop FreeRTOS for radio management purposes?
teamonkey 24 days ago [-]
Tactility appears to be built on top of FreeRTOS.
FreeRTOS itself is very barebones, a library that provides basic memory management, task scheduling, io and a TCP stack, but not, for example, an abstraction layer for screen, keyboard or other peripherals, or the concept of running user applications.
MrBuddyCasino 24 days ago [-]
If you use ESP-IDF, then yes. I‘m not sure other OS support wifi.
dfox 24 days ago [-]
Espressif ships the wifi driver also as an .o file that takes huge struct of function pointers to OS-provided functions that works with other RTOSes. But you need some kind of RTOS.
MrBuddyCasino 24 days ago [-]
Thats progress! Is it already integrated in another RTOS?
RIOT-OS implements shims of the FreeRTOS functions required by the Wifi driver when running on ESP32
anigbrowl 23 days ago [-]
What's the advantage over The device as ap and just flashing it from your development environment? I'm quite interested as I'm into ESP32, but the lack of detail is disappointing and makes me not want to invest time.
ByteWelder 23 days ago [-]
Ease of development: programming WiFi, for example, takes easily 200 lines of code with ESP-IDF SDK. And your credentials will be baked into the ROM as plain text. With Tactility it’s stored safely and all you have to do is toggle the WiFi toggle on.
You can also run multiple apps alongside, making your ESP32 device more of a multitool like Flipper Zero.
jki275 23 days ago [-]
You absolutely do not have to bake wifi creds into an esp32 app in plain text. ESP-IDF provides multiple ways to do this, the way I generally do it is to come up with a wifi AP, serve a web page that allows the user to input credentials, and then store those credentials in encrypted flash. All of this is very easy to do in ESP-IDF using C++.
anigbrowl 22 days ago [-]
your credentials will be baked into the ROM as plain text
what
If you can store this safely...so can other developers. The ESP32 has encrypted flash and critical data (like serials) can optionally be burned permanently without being readable through debug ports etc.
bigfishrunning 24 days ago [-]
I have a few M5 cardputers, i wonder how hard a port would be
ByteWelder 23 days ago [-]
I have been considering that hardware, but the UI isn’t currently fit for such small screens. I’m considering T-Embed first, so when that works, it might be easier to adapt for Cardputer too.
trilbyglens 24 days ago [-]
This is sweet! Looking forward to playing with this
bayindirh 24 days ago [-]
Moreover, it's licensed with GNU/GPLv3, which I love.
In short: just make a CMake build of the project without the ESP32 toolset in your path.
Currently the build was only verified working on Arch/Ubuntu or Win11 with WSL+Ubuntu.
corv 24 days ago [-]
I’m glad to see M5Stack is supported!
vaxman 24 days ago [-]
It is probable the latest editions of ESP32, Rockchip and other controllers manufactured in China do contain CCP exploits. Even the Intel-based mainboards from certain Chinese manufacturers also contain EFI with CCP exploits. (It's not about the chip or brand, it's about the CCP forcing this on China-based manufacturers.) Disclaimer: I've been downvoted over raising this in years past, but now it's a widely covered mainstream issue. I too like the idea of $2 multicore SoCs with integrated RF, but not devices that can potentially blow an on-dye fuse with a specific RF-delivered opcode sequence to enable "new functionality"..and for you technician types, there is a lot more to RF than BLE/WiFi/NFC and various HA protocols that some ESP officially support.)
I saw soon to retire Jensen announce a new SoC (https://www.reuters.com/technology/nvidia-ceo-says-mediatek-...) at CES 2025 in partnership with Taiwan's Mediatek for their new DIGITS PC (just before NVIDIA stock tanked on the speech) and thought his fire is playing with fire, but this may be a solution to untrustable RockChip performance, Qualcomm profiteering and Apple proprietary solutions.
be_erik 24 days ago [-]
Do you have proof of these claims that you can link here on HN? While I’m not surprised, maybe you can provide proof of which brands are working with the CCP and the kinds of backdoors being installed?
vaxman 19 days ago [-]
ALL CHINESE BRANDS ARE WORKING WITH THE CCP. It is literally required by Chinese law!
You can be pointed towards the novel research work that explains how to self-modify the die in the field using opcode sequences that, by the way, came out literally at the dawn of the design timeframe for the latest generation of unnecessary ESP32 chips that flooded the market at the peak of the global chip shortage. The trigger mechanism is not novel. What's more, such n-th level stealth is not even really necessary to conceal backdoors because so many Chinese SoCs ship with reprogrammable microcode and heavily modified Linux distributions that are missing key components despite being marketed with open source trademarks like "OpenWRT". All of this Chinese-fabricated trash is going to be banned for import to US.
Again, half of China, plus their industry and national proxies are out there downvoting away. When will the US learn?
beardyw 24 days ago [-]
An "ESP Microcontroller" definitely doesn't have a keyboard or a screen. That title is misleading and should say something like "ESP based device". I can buy an ESP32 for £3, these are a different thing altogether.
TechSquidTV 24 days ago [-]
If you buy a desktop computer it 100% does not come with a screen and only sometimes comes with a basic keyboard. This doesnt seem relevant.
leptons 24 days ago [-]
> it 100% does not come with a screen
There are plenty of "all-in-one" desktop PCs that are a PC with a built-in screen. Apple, Dell, HP, Lenovo, and others make them.
junon 23 days ago [-]
This seems like needless pedantry.
leptons 23 days ago [-]
This seems like a needless comment.
bayindirh 24 days ago [-]
Yet you can use the same OS for headless application development. No?
Not all x86 systems have keyboards and screen, but Linux and Minix work on them with no problem.
Rendered at 08:22:38 GMT+0000 (Coordinated Universal Time) with Vercel.
Our objective is to build a modern PDA device via a mostly stand-alone watch that can be synced across devices (initially the Linux desktop). We want to achieve tasks that you might typically do on your desktop, focussed towards productivity.
We did consider a custom OS, but decided against it for a few reasons:
1. Allowing somebody else to handle basic OS stuff allows us to concentrate on what really matters, the higher level stuff on top.
2. Having multiple threads in micropython is super simple and we are able to run many active apps at the same time, rather than having to kill them off [3]. Our background apps can continuously interact with the network in the background.
3. Code written for micropython can be easily run on other Python-capable devices.
[1] https://micropython.org/
[2] https://lilygo.cc/products/t-watch-2020-v3
[3] https://tactility.one/#/application-lifecycle
The CCCamp23's flow3rbadge also used micropython to implement its app framework st3m: https://flow3r.garden/
Our system is not all dis-similar from flow3r it seems [2].
[1] https://docs.micropython.org/en/latest/library/_thread.html
[2] https://docs.flow3r.garden/app/guide/basics.html
Hence why unless we're doing some kind of PIC like development, it is about time to embrace more modern tooling. :)
I'd probably just output a middle finger somewhere and halt forever.
Do you think the hardware would be a suitable platform for voice assistant type applications, with AI on server side, of course?
[EDIT] Looks like the T-Watch 2020 also has HA support https://github.com/velijv/LILYGO-T-Watch-S3-ESPHome/tree/mai...
https://tristam.ie/2024/1126/
If you freeze them to save the battery, how do you handle unfreezing?
We can for example put the ESP32 into a light sleep for some time [1] and keep networking alive if apps require it. The idea is to just stretch the battery to the length of a day, shutting down the display gets a lot of the way already.
[1] https://randomnerdtutorials.com/micropython-esp32-deep-sleep...
It's still on a branch, but I compiled and ran it, and now I have two T-decks that can communicate with eachother off-the-grid without a smartphone attached to send messages; it's actually usable in emergencies now, which is why I bought the devices in the first place.
Currently in the process to deploy a mesh from me to my parents and family.
[0]: https://github.com/meshtastic/firmware/pull/3259
The default (and most widely used, including on the de facto public channel) setting is one of the longer range options and gives you around 1 kbps, and it's quite usable on the default for normal texting and positioning telemetry with a handful of active users in my experience. The fastest setting is over 20 kbps and still gets ok range, the slowest is a mere 0.09 kbps and is only useful in some limited scenarios.
It's legitimately useful, and I actually use it regularly. Mostly for hiking, as I do a lot of hikes where we split and meet up. It's also far cheaper than commercial options, and getting close to being as polished as the commercial options like Beartooth or GoTenna.
https://meshtastic.org/docs/overview/radio-settings/
FreeRTOS itself is very barebones, a library that provides basic memory management, task scheduling, io and a TCP stack, but not, for example, an abstraction layer for screen, keyboard or other peripherals, or the concept of running user applications.
You can also run multiple apps alongside, making your ESP32 device more of a multitool like Flipper Zero.
what
If you can store this safely...so can other developers. The ESP32 has encrypted flash and critical data (like serials) can optionally be burned permanently without being readable through debug ports etc.
In short: just make a CMake build of the project without the ESP32 toolset in your path.
Currently the build was only verified working on Arch/Ubuntu or Win11 with WSL+Ubuntu.
I saw soon to retire Jensen announce a new SoC (https://www.reuters.com/technology/nvidia-ceo-says-mediatek-...) at CES 2025 in partnership with Taiwan's Mediatek for their new DIGITS PC (just before NVIDIA stock tanked on the speech) and thought his fire is playing with fire, but this may be a solution to untrustable RockChip performance, Qualcomm profiteering and Apple proprietary solutions.
You can be pointed towards the novel research work that explains how to self-modify the die in the field using opcode sequences that, by the way, came out literally at the dawn of the design timeframe for the latest generation of unnecessary ESP32 chips that flooded the market at the peak of the global chip shortage. The trigger mechanism is not novel. What's more, such n-th level stealth is not even really necessary to conceal backdoors because so many Chinese SoCs ship with reprogrammable microcode and heavily modified Linux distributions that are missing key components despite being marketed with open source trademarks like "OpenWRT". All of this Chinese-fabricated trash is going to be banned for import to US.
Again, half of China, plus their industry and national proxies are out there downvoting away. When will the US learn?
There are plenty of "all-in-one" desktop PCs that are a PC with a built-in screen. Apple, Dell, HP, Lenovo, and others make them.
Not all x86 systems have keyboards and screen, but Linux and Minix work on them with no problem.