welcome AWS to 2018!

Yep. It's pretty boring. I've been using it at home for years and years with libvirt on very not-special consumer hardware. I guess the AWS clown is finally catching up on this one little not-new-at-all thing.

▲

otterley 9 hours ago | parent | next [-]

I was an Amazon EC2 Specialist SA in a prior role, so I know a little about this.

If EC2 were like your home server, you might be right. And an EC2 bare metal instance is the closest approximation to that. On bare metal, you've always been free to run your own VMs, and we had some customers who rolled their own nested VM implementations on it.

But EC2 is not like your home server. There are some nontrivial considerations and requirements to offer nested virtualization at cloud scale:

1. Ensuring virtualized networking (VPC) works with nested VMs as well as with the primary VM

2. Making sure the environment (VMM etc) is sufficiently hardened to meet AWS's incredibly stringent security standards so that nesting doesn't pose unintended threats or weaken EC2's isolation properties. EC2 doesn't use libvirt or an off-the-shelf KVM. See https://youtu.be/cD1mNQ9YbeA?si=hcaZaV2W_hcEIn9L&t=1095 and https://youtu.be/hqqKi3E-oG8?si=liAfollyupYicc_L&t=501

3. Ensuring performance and reliability meets customer standards

4. Building a rock-solid control plane around it all

It's not a trivial matter of flipping a bit.

▲

ssl-3 8 hours ago | parent | next [-]

There's no better way to get good information that is right, than to say something that is misguided and/or wrong.

Thanks for the well-reasoned response.

▲

QuinnyPig 8 hours ago | parent | prev | next [-]

I always enjoy the color you add to these conversations. Thanks!

	▲	sien 7 hours ago \| parent [-]
		I always enjoy the color you add to these conversations in your newsletter. It's provided many a chuckle. Thanks!

▲

PunchyHamster 2 hours ago | parent | prev | next [-]

All that sounds like it would better be a contribution to KVM from the get go rather than invent stuff that eventually showed up in KVM anyway

▲

raw_anon_1111 8 hours ago | parent | prev | next [-]

Seriously curious, don’t Firecracker VMs already run on EC2 instances under the hood when they host Lambda and Fargate?

▲

otterley 8 hours ago | parent | next [-]

Unfortunately I'm not at liberty to dive deep into those details. I will say that Firecracker can be used on bare metal EC2 instances, whether you're a public customer or AWS itself. :-)

▲

raw_anon_1111 8 hours ago | parent [-]

I guess I should have peeked at the source code when I was there…

	▲	rescbr 7 hours ago \| parent [-]
		No need, at least when I was there when the day was still one, before the pandemic. And well, Firecracker is open source. A few of the best technical presentations that I've watched were at a pre-SKO event. Nitro, Graviton and Firecracker. Great engineering pieces, the three of them.

▲

wmf 8 hours ago | parent | prev [-]

Since I don't work for AWS I'm allowed to say that at the scale of millions/billions of microVMs you're better off running them on bare metal instances to avoid the overhead of nested virtualization.

	▲	otterley 7 hours ago \| parent [-]
		I used to work for AWS and I’m allowed to say the same thing. ;-)

▲

sitole 8 hours ago | parent | prev [-]

Nitro is very interesting stuff

▲

tryauuum an hour ago | parent | prev [-]

the only thing I know about nested virtualization is from the libvirt/KVM world too:

* you are right, it just works

* but there were scary notes about the stuff which might happen when you live migrate a virtual machine between hypervisors and the machine has nested virtual machines inside it. I remember the words "neither safe nor secure"