What's the best provider to self-host Firecracker? I feel that AWS is not a safe or cost-effective option for a self-funded startup or small business. Although is anything cost effective anymore? Hetzner just had a massive price hike.

Part of it might just be that I am old and inflation is catching up with my understanding of prices.

But as far as AWS I still have to say no thanks. Imagine some group actually started using my hosted AI agent service for something compute and network intensive. It could turn into $2000 overnight and if I didn't account for one of the numerous types of AWS charges, I might have only collected $500 for credits purchases.

Or it could easily be ten times that. But who am I kidding. No one is going to use my agents. So it doesn't matter if it's gvisor or Firecracker or whatever.

The simplest worthwhile DIY sandbox you can have is to layer two tools: bwrap and gvisor.

    bwrap args -- gvisor args do args -- /path/sandboxee args

Standalone gvisor (not the 'do' subcommand) used to be a mess with the OCI json requirement, but recently they began work on presenting their own bwrap interface (likely to pursue AI agent uses) though I wouldn't use it myself yet.

People often look down on gvisor because they think it's some kind of syscall filter, it is not. It can use one of ptrace, seccomp or even KVM to intercept ALL syscalls and service them with it's own logic (which is in Go). Basically it's a VMM and kernel in one.

Are you looking for highly ephemeral nodes, where you are writing automation that will use the API to orchestrate it? Or do you just want small microVMs that you launch and kill?

Firecracker just has a ReSTful unix socket with a defined API and launches KVM vms with limited options.

For custom SMB I still think libvirt is a lower entry cost and may have transferable use cases to longer lived VMs, so you can just launch a qemu microvm[0] and use virsh and/or libvirt xml to set up the networking.

The ~400ms boot time of a qemu microvm vs ~120ms for firecracker may not be an issue for some loads, but qemu will also allow you a bit more density of placement than firecracker. qemu microvms will use a bit more memory individually, but they will also tend to use less real system memory with a larger number of microVMs.

It is all tradeoffs, and kata containers are yet another option that may apply depending on your use case.

You can run your own firecracker or qemu/kvm microvms on most instances that allow nested hypervisors, or on a local host. If cost containment is critical to you this is one possible way forward.

Really it just depends on if you want/need ReSTful control, or need to support short lived serverless functions, or if CLIs fit better and you many want to support full VMs.

They both are just Virtual Machine Monitors that targeted different use cases and decided on different tradeoffs.

Just be careful about hosting traditional containers and microVMs on the same system, that config is going to be problematic do to fundamental reasons that are too complex to properly address here.

[0] https://www.qemu.org/docs/master/system/i386/microvm.html

Why do you want to self-host vs. using one of the many providers out there?

Daytona, E2B, OpenComputer, Freestyle, Blaxel, Vercel, Modal, Cloudflare, Tensorlake, Superserve, etc. etc.

Some of them work by pre-purchasing credits, so you can control the blast radius of spend.

Also, if you want a more embedded sandbox runtime as a library instead of a daemon + REST API, you can check out libkrun (and friendly layers on top of it like https://microsandbox.dev/ and https://smolmachines.com/)

Hetzner is still cheap compared to AWS.

For self-hosting, have a look at what we're building with SlicerVM.com (disclosure: I'm the founder). Also runs just as well on Apple Silicon.

We run quite a few Slicer instances on mini PCs and Ryzen builds - also on Hetzner (and yes ouch 120 EUR / mo up to ~ 550 EUR / mo for 16core / 128GB RAM feels almost unfair)

This reminds me of Fly.io's model off the top of my head, though its not a self-hosted firecracker as such.

Cloudflare is cost effective for certain types of workloads, I've heard of businesses getting surprisingly far on the $5/mo worker plan.