BuildKit also comes with a lot of pain. Dagger (a set of great interfaces to BuildKit in many languages) is working to remove it. Even their BuildKit maintainers think it's a good idea.

BuildKit is very cool tech, but painful to run at volume

Fun gotchya in BuildKit direct versus Dockerfiles, is the map iteration you loaded those ENV vars into consistent? No, that's why your cache keeps getting busted. You can't do this in the linear Dockerfile

▲

kodama-lens 38 minutes ago | parent [-]

I switched our entire container build setup to buildkit. No kaniko, no buildah, no dind. The great part is that you can split buildkitd and the buildctl.

Everything runs in its own docker runner. New buildkitd service for every job. Caching only via buildkit native cache export. Output format oci image compressed with zstd. Works pretty great so far, same or faster builds and we now create multi arch images. All on rootless runners by the way

	▲	verdverm 10 minutes ago \| parent [-]
		That's pretty cool, rootless would be nice, but more effort than we see in ROI currently. I'm using the Dagger SDK directly, no CLI or modules. Had to recently make it so multiple versions can run on the same host, such that as developers change branches, which may be on different IaC'd versions (we launch on demand), we don't break LTS release branches.