| ▲ | Show HN: The Unite real time operating system(jacquesmattheij.com) |
| 138 points by jacquesm 5 days ago | 46 comments |
| |
|
| ▲ | jacquesm 2 hours ago | parent | next [-] |
| If you want to run this under windows then VirtualBox seems to work better than QEMU. |
|
| ▲ | whitehexagon 8 hours ago | parent | prev | next [-] |
| >dive under the table every three minutes to hit the reset button Hah, I feel that pain. Currently hacking some bare-metal Zig on PinePhone, and on top of the USB reconnect power cycle, I am transferring an SD card back n forth. I wonder if I'll be making a similar post in 30 years time. Anyway, well done, and I look forward to browsing the code for some inspiration. Thanks for making it PD. |
| |
| ▲ | jacquesm 3 hours ago | parent [-] | | Oh good luck, I really feel your pain :) Consider using an emulator until you're really ready to go to hardware that might save you a lot of time and headache, it also makes observability a lot better. Nothing beats having access to memory from a debugger. | | |
| ▲ | whitehexagon 31 minutes ago | parent [-] | | Thank-you for the advice. So far it has been mostly SoC stuff, and I recently got side-tracked learning some armv8 assembly, which reminds me somewhat of my 68000 days and has been good fun. I did try QEMU for the first time last week, but started thinking I'd need some way to mock out all the hardware, and I only got as far as UART. But now that you mention it I'll take another look. There is a lot of multi-core mbox? IRQ stuff I'm about to tackle next, and I dont see why I cant develop all of that independent of the hardware. I seem to be learning the hard way. I spent the first 12 months debugging via the only thing I could get working, an onboard RGB LED. |
|
|
|
| ▲ | Rochus 13 hours ago | parent | prev | next [-] |
| That's interesting. Did you have access to the QNX sources at the time and were able to study the QNX kernel and why it had a much better performance than Minix? Given your reference to DJGPP, am I right to assume that you have written your OS in C? (I wasn't able to download the source code and would appreciate to be able to download a compressed file from a "normal" web server). Since you made comparisons with the Minix version of that time, what conclusions did you draw from this, and what especially made your kernel "lightyears ahead" of Minix? |
| |
| ▲ | jacquesm 3 hours ago | parent [-] | | No, but I had used it extensively and written a fairly large amount of code for it so it was mostly re-implementing how I thought things must work under the hood. The hardest part was to bridge some of the posix like functionality to this rather strange set of primitives, especially 'fork' and the '/dev/' parts required some pretty hacky solutions. But they work and they ended up being quite reliable. As for it being lightyears ahead of Minix: Minix was built from the ground up as a platform to demonstrate how Unix works without being Unix. QnX and by extension my version on that theme was built from day one to be low latency and to have the smallest possible kernel footprint that you could have. The lack of a root file system and the Plan-9 like network transparency are all as a result of that and allow extremely fine grained control over what goes into the final package. It is far more suitable for low level control while at the same time having high level tooling than Minix. What I loved about Minix is the way the source code is organized and how - and this is obviously no accident - it reads as a textbook, I learned a lot from looking at the code and ended up adopting the file system as a stand-alone piece of code (which allowed me to pass files in and out of the OS early on by writing them under one OS and then rebooting so I had access on the other side). | | |
| ▲ | Rochus 3 hours ago | parent [-] | | Amazing, thanks. Is it possible to just have a look at the source code somewhere, e.g. codeberg or gitlab? I'm especially interested in the microkernel IPC. | | |
| ▲ | jacquesm 3 hours ago | parent [-] | | The send/receive/reply bits are almost trivial, I can mail you the relevant parts. I am explicitly not putting any of this on GitLab or any other medium that is easily ingested by the AI peddlers, if they want my code they'll have to work for it :) If you drop me a line on jacques@modularcompany.com I'll send you back a zip file. | | |
| ▲ | Rochus 2 hours ago | parent [-] | | In QNX, the (async) IPC is integrated with the scheduler, which is apparently responsible for the performance (still researching). So I was wondering if you chose a similar architecture. My email is "me at rochus-keller dot ch". Thanks. | | |
|
|
|
|
|
| ▲ | Animats 15 hours ago | parent | prev | next [-] |
| Aw. I've always liked QNX. But a modern implementation, not a revival of a 30 year old abandoned project, would be more useful. We really need something for small IoT boxes that has less attack surface than Linux. The problem people run into trying to build something like QNX is that, while the kernel is tiny, it won't even do Hello World until you have the necessary user-space drivers running. Initial debugging is hard. |
| |
| ▲ | jacquesm 3 hours ago | parent | next [-] | | QnX is expensive. And in a way that is exactly what I'm hoping for with this release: that someone will pick it up, see how simple a robust OS can be and to use it as a base to build on (for instance, by porting Rust and rewriting the core in that, which would give a very quick path to a self hosting OS), or maybe just as a source of inspiration. Here you have all of the scaffolding in place already so now you can focus on those other parts one at the time while having the luxury of good tooling and a nice debugging environment, no need to bring it up from scratch. Another great thing would be to make an ARM or other CPU port. And that's not all that hard: there is only one process that is CPU specific and there are a very limited number of pieces that you need (essentially just a COM port driver) to be able to make it self hosting. | |
| ▲ | glonq 6 hours ago | parent | prev | next [-] | | Back when I did embedded development full-time I always lusted after QNX, but the costs and royalties were higher than what our low-volume product could justify. Or maybe we were just cheapskates. | | |
| ▲ | jacquesm 3 hours ago | parent [-] | | No, you are right, in volume QNX is expensive. We spent a ton of money ($1600 / license iirc) on licenses for a system that we built, and even so it felt like Quantum never took us serious because we only needed 10's of them rather than the 100's of thousands they were used to selling to their bigger automotive customers. But it is still an OS that will for me always embody the 'road not taken' and I still strongly believe that it was a better road, based on my experience with IRIX, Windows, DOS, VMS, Linux, VAX Unix, OS/9, Plan 9 and QNX. It felt like that only one that always ran and that always stayed responsive, no matter what we threw at it. Across many years and 10's of machines running various versions of QNX I have not seen a single production crash of the OS that was not an indication that the underlying hardware had died. But it was perfectly possible to anticipate even that and have an Erlang like supervisor system in place to deal with that. |
| |
| ▲ | bregma 12 hours ago | parent | prev | next [-] | | If you're looking for something like QNX you could use... QNX. It's still around and free for hobbyists [0]. You can use it knowing you're using the same OS as your car, or train, or medical device, or spacecraft. [0] https://www.qnx.com/products/everywhere/ | | |
| ▲ | p_l 8 hours ago | parent | next [-] | | After previous rugpull, why anyone would want to jump through the hoops? | | |
| ▲ | Animats 4 hours ago | parent [-] | | Yes. The history is awful. Closed source, partially open source with free version, closed source, fully open source with free version (but not for commercial use), and then, suddenly one day, closed source. Twenty years ago, many of the Gnu tools built for QNX by default. That stopped. You can get a "personal use license" now, but you can't distribute anything that has parts of QNX code in it. | | |
| ▲ | p_l 2 hours ago | parent | next [-] | | From what people reported around the rugpull, BB pretty much nuked all customer relationships with various groups that had commercial licenses too, not just with people who looked into open source or free-as-in-beer options. Greatly accelerated AGL with D-Bus (yuck) as patchwork replacement for QNX IPC | | |
| ▲ | jacquesm 2 hours ago | parent [-] | | Yes, this happened to us as well. It basically killed more than a decade of development. I did write a send/receive/reply/name_attach/name_locate library for Linux, which worked well enough that we could at least rescue the project. But BB killed QnX for large scale software development projects outside of the embedded space, and quite possibly for a lot of those as well (but I had no contact with such groups). There was a point in time where QnX ran a very large fraction of all of the world's infrastructure and BB showed the dangers of relying on a company that never fully committed to their long-term strategy. | | |
| ▲ | p_l an hour ago | parent [-] | | Some of the older autonomous challenges involving flight I remember a lot of contestants had QNX on board vs. let's say VxWorks or others. I also guess QNX having more point-to-point IPC/RPC fits some stuff better without having to plop in (possibly expensive) middleware like CORBA-RT |
|
| |
| ▲ | jacquesm 3 hours ago | parent | prev [-] | | All of the acquisitions and license issues did QnX no favors. We built a very large installation on top of it and always felt that Quantum didn't really know what they wanted to be. The BB acquisition was the final nail in the coffin, though, as a platform it was a pretty good illustration of how powerful that whole mechanism is. |
|
| |
| ▲ | Chris2048 10 hours ago | parent | prev [-] | | You still have to register with them, I'm not sure that is common among NC licenses. I assume this also means its not really open-source (or compatible with open collaboration) if they are gatekeeping (via licences) the free dev licence. Also consider this - If you got a licence would this "taint" you if you ever wanted to work on a project like OPs? As in, the registration is proof you could access the proprietary QNX code, forming the basis of a lawsuit. | | |
| ▲ | jacquesm 3 hours ago | parent [-] | | Quantum has muddied the waters considerably here by at some point releasing the code under a fully open license. They tried hard to reverse that decision but I'm sure there are still copies of that version floating around. |
|
| |
| ▲ | naasking 8 hours ago | parent | prev [-] | | > We really need something for small IoT boxes that has less attack surface than Linux. seL4 exists. | | |
| ▲ | Animats 4 hours ago | parent [-] | | People mostly use seL4 as a hypervisor, running Linux on top of it. QNX has a POSIX API, so you don't need another layer. | | |
| ▲ | naasking 3 hours ago | parent [-] | | QNX is allegedly a microkernel. Either the QNX POSIX API is itself another layer over the microkernel, or QNX cannot be a microkernel (hint: it's the former [1]). People run L4 as a hypervisor only because they want access to the full Linux API. That's not really what you're talking about though, and there's nothing stopping a thin QNX-style POSIX compatibility layer as a user-space server or library as with QNX. The point being, the mature, widely deployed, robust and widely tested microkernel you're looking for already exists, so you just need to focus on the POSIX layer. [1] https://www.qnx.com/developers/docs/8.0/com.qnx.doc.neutrino... | | |
| ▲ | Animats 2 hours ago | parent [-] | | Actually, most of the POSIX compatibility stuff is in QNX's equivalent of "libc". When you call Unix "write", the library does a MsgSend to the appropriate service. MsgSend is in the kernel, but whatever performs the write is in some other process. Some other functions, such as program loading, are in user space .so files. When you spawn a new process (in QNX, spawn is the usual function, and fork/exec are emulated for backwards compatibility), the library in your program attaches to the .so file that contains the program loader. The user process requests memory and decodes the executable image, and sets up the new process environment. If something in executable file decode does something wrong, the process faults in the usual user space way. There's no privileged emulation layer. At boot, the .so files for this sort of thing are part of the boot image and are loaded into memory, so they are available even before there's a file system. As are the userspace programs needed during startup. You can build your own boot image, with whatever programs you want. You might, for example, include the "dashboard driver" for a car, and omit terminal support.
There's no assumption in QNX that there's a console, because, often, there isn't. | | |
| ▲ | jacquesm 2 hours ago | parent [-] | | That's exactly how I did it too. The biggest hack for me was the 'open' call when using a device. That first does a lookup for a service with the name following the /dev/ bit and then it redirects the 'open' message to that process rather than to the active drive's filesystem. This allows for the appearance of a /dev/ directory without it actually existing anywhere. This makes some Unix like mechanisms hard to implement (for instance, you can't just do 'ls /dev/') but it made enough stuff work out of the box that it seemed worth the trade-off of bypassing the filesystem. It also makes the file system optional, which can come in very handy, because that means no file system is every privileged in the sense of a Unix '/' filesystem. They're all equal and you can kill each and every one of them and still have a running system. |
|
|
|
|
|
|
| ▲ | alain94040 17 hours ago | parent | prev | next [-] |
| Interesting. Is there a picture that explains how the layers talk to each other?
Is there a VM system? How does message passing work, what kinds of protections are between tasks? I have had many ideas for such kernels over the years, but not the patience yet to implement any of them. I wonder if Claude could help me with generating a kernel in less than a day. |
| |
| ▲ | jacquesm 3 hours ago | parent | next [-] | | I should definitely make that picture. Yes, it as VM. Message passing all runs through the kernel. Task isolation is strictly memory based, whether a task acts on or responds to a message is something the tasks will have to negotiate between themselves. You could whitelist, use a token scheme or any other method for authentication that you wish to put in place, but out of the box there is none, if you know a tasks ID you can send it a message. Obviously there are ownership of resource constraints and the kernel will always ensure that the receiving task knows who the sending task is to ensure that parties are not stealing each others descriptors and such. | |
| ▲ | kroaton 11 hours ago | parent | prev [-] | | Pull the source and ask Claude to explain it to you... |
|
|
| ▲ | AndrewKemendo 4 days ago | parent | prev | next [-] |
| Congrats on shipping Jaques! Great to see you resurrect this project and publish it. |
| |
| ▲ | jacquesm 4 days ago | parent [-] | | Thank you. One download so far :) Mega success! It's interesting how you can release an OS + source code and hardly anybody will bat an eye. But nonsense attached to an LLM will draw a crowd ;) Anyway, I'm happy that I got it to work and I hope at least someone will find it useful. | | |
| ▲ | balder1991 3 days ago | parent | next [-] | | Maybe he should have named the OS the “LLM OS”, and then only later people will realize LLM stands for something else entirely. | | |
| ▲ | jacquesm 3 days ago | parent [-] | | That's slightly evil, but funny too! | | |
| ▲ | jitl 9 hours ago | parent [-] | | You just need a network stack, http client, and json parser and your os can have a full fledged LLM system in the kernel. Raise 10M seed and off to the races you go! | | |
| ▲ | jacquesm 3 hours ago | parent [-] | | Hehe. Indeed. So that isn't happening. The last thing I need is some VC breathing down my neck. |
|
|
| |
| ▲ | AndrewKemendo 4 days ago | parent | prev [-] | | Just gotta ride the wave as you know. Sorry I misspelled your name originally! | | |
|
|
|
| ▲ | bcjordan 3 days ago | parent | prev | next [-] |
| Really interesting. Feels like a fun platform to tinker with like "circuit bending" to make surprising behaviors happen across the entire OS I wonder what the ideal demo of this that makes its approach visually legible would be |
| |
| ▲ | jacquesm 3 days ago | parent [-] | | That's a good question and one that I've been wondering about myself. For me it is the very clean abstraction layers. There is zero side-to-side communication it is all broader foundations to narrower verticals plugged in to those. For instance, the networking stack has a process for every layer. That keeps that whole thing manageable and it means that problems can never escape the scope of the process they are in, unlike a 'large' kernel where a botched device driver can bring down the whole system. You can pretty much tear this down to the task scheduler and the loader and rebuild it up without taking the system down. And I had a good idea on how you could replace the scheduler with another one while the system is running. I'm really curious what - if anything - people will do with it, I think that the fact that an OS like this one powered the BlackBerry (which people loved) should work in its favor, other places where you find this kind of OS is in industrial control and in vehicles. Usually there is some kind of hardware component involved but that's entirely optional, you could use it as a general purpose OS as well. |
|
|
| ▲ | landsman 2 days ago | parent | prev [-] |
| I would consider putting it on GitHub. Torrent is not the best way to share code today. |
| |
| ▲ | jacquesm 2 days ago | parent [-] | | It's a running OS snapshot as a drive image, not a source tree, the sources are on the drive image. As for GitHub itself: I am not going to further Microsoft's grip on the world of software development. | | |
| ▲ | ThinkBeat 9 hours ago | parent | next [-] | | Thank you. For taking a stand against Microsofts
"social network", "ai training data", greed
and proprietary tooling. I think sharing the source code as you do now
is a perfect choice is it is what you are most
comfortable with. | | |
| ▲ | jacquesm 3 hours ago | parent [-] | | I may end up working some more on it and releasing updated versions of the torrent. But I'm not going to be a 'maintainer', if I want unpaid work I'll volunteer at the local animal shelter instead. |
| |
| ▲ | ofrzeta 17 hours ago | parent | prev [-] | | Well, then go to codeberg.org |
|
|