| |
| ▲ | immibis2 10 minutes ago | parent | next [-] | | I think the use case is that you buy a VM from someone and you also want to run VMs. You wouldn't design your own infrastructure to run VMs inside VMs, but you might already be inside a VM, and want to run VMs. | |
| ▲ | JoshTriplett 2 hours ago | parent | prev [-] | | > I have yet to hear a compelling use case for nested virtualization. AWS instances are slow to launch; they take whole seconds. Nested virtualization allows running something like firecracker/crosvm/rust-vmm atop a normal AWS instance, without needing a bare-metal one. | | |
| ▲ | tptacek 2 hours ago | parent [-] | | If you're segregating internal workloads, by all means use nested virtualization on EC2 or DO or something. Your threat model is different and you probably have a lot more room to maneuver when a KVM escape is announced, because you have some level of trust in all your workloads. If you're running multitenant for mutually-untrusting customers, that excuse goes out the window. | | |
| ▲ | JoshTriplett 2 hours ago | parent [-] | | One distinction I'd make as well is "are you running more than one customer's workload on the same system" (yikes, don't), vs "are you using nested virt as additional protection rather than running the customer workload directly on an instance, but one instance only gets jobs from one customer". | | |
| ▲ | tptacek 2 hours ago | parent | next [-] | | Another good point, but this one is maybe a little less dispositive than it looks like: it's very difficult to engineer an orchestration system that reliably segregates worker physicals from each other, so I'd rate the likelihood of a game-over situation after a guest->host escape as very high regardless of how yuo schedule workloads. It's sort of the same situation as how SSRF's are usually game-overs (you're pivoting from a hardened attack surface to a much softer one) but even harder to engineer again. | | |
| ▲ | JoshTriplett 2 hours ago | parent [-] | | That's fair. You'd want a system designed around the notion that every customer instance must be isolated from each other without the nested virtualization. (Aside: my current guess at the appropriate security posture towards Nitro Sandboxes, in the current era of vulnerabilities, is "decent, but probably has security holes too, and is less widely scrutinized, don't count on it either". Thoughts?) Any recommendations for good guides on that, besides "design very careful IAM policies and security groups, harden any common services they have access to, and ideally run them under separate AWS accounts"? | | |
| |
| ▲ | necovek an hour ago | parent | prev [-] | | Does this hold for the likes of Amazon? Do your lambda functions always get a per-customer dedicated instance or is firecracker distributing them so that multiple customers can have VMs on the same actual box? Sounds like it would be pretty expensive for Amazon to dedicate an instance to every customer, especially for those with small workloads. If that's the case, why would it not hold for Amazon but would hold for everyone else? | | |
| ▲ | swiftcoder an hour ago | parent [-] | | > Do your lambda functions always get a per-customer dedicated instance At some point in the past they did, for pretty much exactly this reason. I'd assume they don't anymore (I last worked there a decade ago). |
|
|
|
|
|