Setting up your own is not that hard and if you bought some compute before the Altman squeeze, very cheap.

Why isn't libkrun good enough for hosted stuff? I use it as a podman backend in a microservice architecture.

What people aren't getting with `firecracker` is utilization. Don't get me wrong, `firecracker` is great software and it's what I'm using for lightweight virtualization, but workloads are really bursty over really short periods of time now, even with the snapshot and restore that you can get if you're willing to hack on `firecracker` substantially, you hit walls where it's like, this is too much against the grain, this thing wasn't designed to bounce from 1 core to 32 to 8 to 16 to 4 to 32 to 1 seamlessly, and that's what it takes to get extreme utilization even with extremely good ML on the prediction.

I am quite sure I'm not the only person working on post-firecracker KVM.

Thanks for sharing these!

AWS AgentCore runtime has been around for about a year: https://docs.aws.amazon.com/bedrock-agentcore/latest/devguid... (spoiler, it's the same underlying technology as the Lambda MicroVMs).

Agreed.

Most of the startups are just wrappers around AWS and significantly more expensive.

Agents need sandboxes that are cheaper so that they can run thousands

I feel that AWS, GCP and all the other cloud providers can provide this natively.

But still it would be nice to self host.

The best part of self hosting is that you own it as well, no rug pulls from the laundry list of reselling providers that could go away at anytime.

It would be nice to have a one click sandbox agent on a self hosted instance that is, free, fast (can pay a bit more for more intensive operations) and that is open source.

Are you looking for highly ephemeral nodes, where you are writing automation that will use the API to orchestrate it? Or do you just want small microVMs that you launch and kill?

Firecracker just has a ReSTful unix socket with a defined API and launches KVM vms with limited options.

For custom SMB I still think libvirt is a lower entry cost and may have transferable use cases to longer lived VMs, so you can just launch a qemu microvm[0] and use virsh and/or libvirt xml to set up the networking.

The ~400ms boot time of a qemu microvm vs ~120ms for firecracker may not be an issue for some loads, but qemu will also allow you a bit more density of placement than firecracker. qemu microvms will use a bit more memory individually, but they will also tend to use less real system memory with a larger number of microVMs.

It is all tradeoffs, and kata containers are yet another option that may apply depending on your use case.

You can run your own firecracker or qemu/kvm microvms on most instances that allow nested hypervisors, or on a local host. If cost containment is critical to you this is one possible way forward.

Really it just depends on if you want/need ReSTful control, or need to support short lived serverless functions, or if CLIs fit better and you many want to support full VMs.

They both are just Virtual Machine Monitors that targeted different use cases and decided on different tradeoffs.

Just be careful about hosting traditional containers and microVMs on the same system, that config is going to be problematic do to fundamental reasons that are too complex to properly address here.

[0] https://www.qemu.org/docs/master/system/i386/microvm.html

Why do you want to self-host vs. using one of the many providers out there?

Daytona, E2B, OpenComputer, Freestyle, Blaxel, Vercel, Modal, Cloudflare, Tensorlake, Superserve, etc. etc.

Some of them work by pre-purchasing credits, so you can control the blast radius of spend.

Also, if you want a more embedded sandbox runtime as a library instead of a daemon + REST API, you can check out libkrun (and friendly layers on top of it like https://microsandbox.dev/ and https://smolmachines.com/)

Hetzner is still cheap compared to AWS.

For self-hosting, have a look at what we're building with SlicerVM.com (disclosure: I'm the founder). Also runs just as well on Apple Silicon.

We run quite a few Slicer instances on mini PCs and Ryzen builds - also on Hetzner (and yes ouch 120 EUR / mo up to ~ 550 EUR / mo for 16core / 128GB RAM feels almost unfair)

This reminds me of Fly.io's model off the top of my head, though its not a self-hosted firecracker as such.

Cloudflare is cost effective for certain types of workloads, I've heard of businesses getting surprisingly far on the $5/mo worker plan.

You absolutely can run agents on a regular VM. But if you want to build multi-tenant and multi-agent systems with strong security boundaries, then having a VM or MicroVM per agent session (or session with a group of agents) really simplifies things.

When we did AWS AgentCore Runtime last year we introduced session isolation, with MicroVMs per session. You can think of Lambda MicroVMs as the same stack, but generalized to fit a larger number of application patterns.

Fargate does not use Firecracker, it is simply ec2 instances.

Yeah, I'm surprised Justin posted this like it was new(s). Wasn't it doing the rounds on the 22nd when it launched?

It just a time limit of the life of a single MicroVM.

Using this for a long lived "developer environment" would be extraordinarily expensive anyhow. Scaling the vCPU + RAM cost of these to the same shape compute optimized Graviton On-Demand EC2 instance (16 vCPU x 32 GB RAM) shows about 4x the cost.

So don't do that. Just use an EC2 instance.

They are long-lived if you're a mayfly.

But I think the point is that they should be cheap to set up, and because of the short life, never really contain anything except the potential to compute when needed, not important data.

You can use them for dev environments.

You just have to finish development in 8 hours.

lambdas are ephemeral on compute, but couldn't you connect up EFS for your long lived data?

then when you launch the next one, its like you are still there?

I'm assuming you can launch them again after 8 hours.

> Didn't mean to highjack for self advertisement. > > As the topic matches, .... my project might be appealing to some here

That's exactly what you intended to do. That is the definition of advertising. It is true, many people might like it, so own it. Don't lie about it, even to yourself.

I tried this a few days ago. Once you have an image built and ready startup time is fast, but building that original image took 5-10 minutes.

I think it's designed for building an image once and then reusing it many, many times.

This is for people who want both faster execution, and better security isolation for agents/subagents. It is a different use case than yours

Presumably it is Firecracker. It's just a different shape of offering, along with Lambda and Fargate, which are also Firecracker.

The literal first paragraph has a highlighted link that says this runs on Firecracker

It's a product that runs on top of Firecracker.

Fly.io doesn't set a maximum of 8 hours of alive time on your instance.

Also, MicroVMs can't be exposed directly to the web. Your code running in them can only be executed via API calls with attached auth tokens - so if you wanted to host a public facing API or website with them you'd need to implement your own additional layer in front.

Something I appreciate about Fly (disclaimer: they support my work) is that the pricing is fixed - you pay $1.94/month (less if you suspend your machine) for the smallest instance, up to $976.25/month for the largest (16 CPUs, 128GB) plus predictable costs for volume storage.

The only variable outside your control is bandwidth, and that's unlikely to cause a nasty shock.

Contrast with any of the more "elastic" hosting providers - Vercel, Cloud Run - and you're much less likely to get a horrifying bill if something gets overly-crawled or goes viral.

e2b supports UDP and the pricing structure is different.

i’d say what AWS released looks closer to a bare compute primitive. E2B is up the stack and ships everything around VM like snapshots, networking, integrations.

also, there’s no lock-in, E2B is open-source and can be hosted on any cloud (AWS included).

plus supports bigger boxes, higher concurrency, longer timeouts (24hr).

disclaimer: i work at E2B

check this out https://github.com/smol-machines/smolvm

will have a hosted platform soon with GPU support (vulkan)

It is supposed to be a sandbox that you can invoke from agent, langchains of the world, coding agents etc.

No, it doesn’t seem like it.

Not that I can find in the docs anywhere. Compute only.