Remix.run Logo
adrian_b a day ago

There is no need whatsoever to dedicate a core for each process which owns peripherals and which services the requests for them.

All the I/O requests would be submitted in queues and whenever a queue would be empty the server process would sleep by invoking a wait for event syscall, like futex_wait of Linux. Obviously, when a device driver sleeps and must wake up, that involves context changes, but this will happen only when the peripheral is seldom used, so the performance loss is not important. Once the device driver process is active, no context changes are necessary for data transfers between processes. When there is heavy I/O activity, a few cores may be occupied with it, but they will always correspond to the active I/O subsystems, not to any that exist but are not currently in use. The bulk data transfers should be done by DMA directly from the user buffers, so the device driver processes should only orchestrate all the transfers, which should not consume much CPU time.

Nonetheless, I believe that the performance could be improved by reserving a single core for the microkernel, for hardware interrupt handling and also for the keyboard and graphic pointer drivers. Only in very big systems with hundreds of cores it might be necessary to reserve more than 1 core.

A user API like that of liburing of Linux can be reimplemented almost with no changes in a system with a micro-kernel. That means that a user application would have an AIO completion queue, which instead of receiving data from the Linux kernel would multiplex the completion messages from all the device driver processes.

For the opposite direction, from a user application to the device driver processes, the implementation of the I/O request submission queue would be a little more complex, because of security requirements. Processes that trust each other, e.g. the children of a single process, could inherit and use a common submission queue. Otherwise, the submission queues must be distinct, i.e. in mapped pages that are shared with the device driver processes, but not between the user processes. Because polling multiple queues might waste time in a device driver process, it is likely that submitting a batch of I/O requests should be done by writing the shared buffers with data, then invoking a syscall similar to io_uring_enter, which would alert the corresponding device driver processes (by posting the request in single queues monitored by each device driver process for new events, but the posting of this small message would be done by the micro-kernel, so that rogue processes cannot mess with it).

Simpler but less performant I/O APIs, like the stdio of libc, can be implemented on top of a multiplex AIO API, like that of liburing.