I agree that virtualization has seen great advances. Kata containers on k8s are almost (not quite 100%) drop in replacement. Regardless those last 10% remain a problem.

I run a personal server for few open source applications for personal use. I was thinking with all the supply chain attacks, and how carelessly I run `docker pull`s to update things I should probably consider hardening things a bit. I thought before jumping to full virtualization with Kata I can easily try gvisor/runsc first. Only to realize that DNS resolution is completely different with runsc vs runc and had to switch back.

Another sticking issue with virtualization is resource allocation. With namespace docker you can easily oversubscribe each container CPU/memory and rely on the single kernel letting individual containers burst as needed. With full virtualization this is still a big problem. Even with balloon devices and dynamic memory and CPU etc, the resource allocation is still not optimal. On a basic 8 core/16GB machine you can run 1 or 2 dozen services and things generally workout fine. Trying to run each of those in a virtualized VM you suddly can maybe run 6 or 7 maybe. There is no way to tell VM 3 kernel to drop its file system cache because VM 6 needs to load a large file in memory. Even if you script it out, now VM 3 is slow because it dropped all its cache while VM 6 finished processing 3 hours ago. These are not unsolvable problems, but despite how far virtualization has come, are still friction points.

Not to mention issues like sharing hardware devices (GPUs, disks, USB devices etc) between multiple VMs