This isn't too dissimilar to DOS programs. DOS also gave you a filesystem, memory management, a shell a process loder, driver interface, but it was all in real mode, and you were free to replace these components with your own as you pleased.

Or if you're more familiar with embedded, like a BSP or RTOS

Arbitrating access to the hardware between multiple tenants still gonna be a PITA (as it was during the DOS days), especially if you want to expose actual hardware, not some generic VirtIO devices. Even with the latter, the hypervisor would have to do quite a bit of the resource management and interface translation.

Unless you're literally running a single application on a dedicated piece of hardware in which case... yeah, sure, you can do the OS-as-a-library schtick. But you could do it 10 years ago, too.