Remix.run Logo
trebligdivad a day ago

Nested virt on x86 is curiously painful; you'd kind of think each layer would be isolated, so that the L0 (hardware) would only have to worry about it's VM (L1), and L1 would have to worry about it's VM (L2); but nope - the L0 top level hypervisor sees faults from the L2 and has to figure out that they are actually L2 not L0. IMHO the extra complexity (and historical flakiness of it) - makes me say that enabling nesting is a bad idea for public VM hosts.

20 hours ago | parent | next [-]
[deleted]
stinkbeetle 17 hours ago | parent | prev | next [-]

The alternatives are the L0 emulates a hypervisor-privileged mode to the L1 which would be a performance cost (and does not really avoid the problem since the L0 would have to emulate vmenter/vmrun and therefore be aware of the L2 anyway, or the hardware would have to provide some nested virtualization facility which seems like it would be complicated and expensive for little benefit, though I could be mistaken.

Does anything do a "real" nested virtualization in hardware? s390 might but I know ~nothing about it and it probably does not expose its bare metal hardware layers to Linux/KVM anyway.

nijave 15 hours ago | parent | next [-]

IBM mainframes might but I have a very shallow understanding.

Found this https://sixe.eu/news/kvm-nested-vms-ibm-power-linux-lpar

bonzini 15 hours ago | parent [-]

That's exactly the same as x86. Nested virtualization support is almost entirely in the hypervisor.

nijave 12 hours ago | parent [-]

x86 doesn't have a firmware hypervisor that allows splitting into LPARs.

That's showing 3 layers of nesting: LPAR, hypervisor, nested hypervisor

bonzini 5 hours ago | parent [-]

On POWER, LPARs and the first level of hypervisor do not use any LPAR-specific nesting support in the processor. It's all handled by the firmware, not the hardware.

bonzini 15 hours ago | parent | prev [-]

No, even s390 needs shadow paging.

Intel and AMD both have some small amount of acceleration of nested virtualization, respectively with shadow VMCS and virtualized VMLOAD/VMSAVE.

iririririr 18 hours ago | parent | prev [-]

but that's the entire point of kvm. that leak is the feature! that's how you get the performance boost.