> I wanted specifically to find a minimal set of restrictions to run untrusted code.

I don't think we should consider containers to be a security boundary. Even full VMs can be escaped, and have been, many times.

The fact that this is possible in the first place makes me think we need a much better approach.

▲

zamadatix an hour ago | parent | next [-]

We don't have any "security boundaries" by this definition, just "security make-it-harder"s. I.e. "security boundaries" always have a relative strength associated with them, not a guarantee they keep the thing secure without any doubts.

▲

chubot 4 hours ago | parent | prev | next [-]

As far as I know, Firecracker, gVisor, and Kata Containers are the solution here. They use VM primitives (x64_64 and ARM64 extensions) and have lighter codebases

https://firecracker-microvm.github.io/

https://gvisor.dev/

https://katacontainers.io/

But I don't have any direct experience with any of them. I'd be curious what people who have built on top of them think

edit: OK it looks like Kata can use Firecracker, so as far as isolation, it's either Firecracker or gVisor. And Firecracker is the VMM I mentioned, but gVisor is quite different -- it's more like a user space kernel that emulates syscalls.

	▲	binsquare 4 hours ago \| parent \| next [-]
		I'm going to toss in smolvm as well because firecracker needs some expertise to make the box usable and secure. https://github.com/smol-machines/smolvm
	▲	johnsmith1840 2 hours ago \| parent \| prev \| next [-]
		I've deploy gvisor, done basic test of firecracker and an honest attempt at production kata. Firecracker and gvisor are nice systems not horrible to use, gvisor isn't quite the same security level though. Kata is HARD to make. The technical know how to make that in production is awe inspiring. I wanted to use it but it was so complicated to integrate into a cluster I literally just gave up and mirrored raw VMs into the cluster which was alot easier actually. Kata also breaks any potential of confidential VM unless you're a virtualization wizard. You should go check out redhat's confidential container method for a production design overview. Their ARO self hosted system.
	▲	laurencerowe 4 hours ago \| parent \| prev [-]
		As I understand it Kata supports multiple VMM backends, Firecracker, QEmu, Cloud Hypervisor, and their own Dragonball. Except QEmu, I believe those are all built on crates in the rust-vmm ecosystem, each making slightly different tradeoffs.

▲

bityard 3 hours ago | parent | prev | next [-]

Depends on the threat model. Security is not black-and-white.

Containers protect against "I don't trust this curlpipe to not crap all over my dotfiles," rather than, "there might be a sandbox escape attack in this random file I downloaded."

If a VM is not sufficient for your threat model, I'm curious what is?

▲

stryan 3 hours ago | parent | prev | next [-]

Podman supports using KVM backed virtualization for containers via libkrun: `podman run --runtime=krun` . Still not the end-all-be-all security boundary, but better I think.

▲

raesene9 4 hours ago | parent | prev | next [-]

I definitely wouldn't trust standard Linux style containers that expose a shared Linux kernel at the moment, there's been far too many LPE and container breakout vulnerabilities this year. It's possible that in future if the kernel gets a lot more hardened, that could change but things like Firecracker are a better bet from a security standpoint.

▲

LtWorf an hour ago | parent | prev | next [-]

They are a security boundary, but like everything else, not perfect.

▲

mdspan 3 hours ago | parent | prev [-]

I think until something hardware-based like CHERI becomes widely deployed (which seems extremely unlikely in the near to mid term given), we're going to keep seeing VM escape CVEs pop up indefinitely.

	▲	LtWorf 12 minutes ago \| parent [-]
		Because we've never encountered hardware bugs…