What were the biggest challenges in terms of designing the VM to have subsecond start times? And what are the current bottlenecks for deceasing the start time even further?

No special programming tricks were used.

Linux was built in the 90s. Hardware improved more than a 1000x. Linux virtual machine startup times stayed relatively the same.

Turns out we kept adding junk to the linux kernel + bootup operations.

So all I did was cut and remove unnecessary parts until it still worked.

This ended up also getting boot up times to under 1s. The kernel changes are the 10 commits I made, you can verify here: https://github.com/smol-machines/libkrunfw

There's probably more fat to cut to be honest.