The layers are tar files, I’m confused what behavior you actually want that isn’t supported.

The original Docker (and the current Podman) created each layer as an overlay filesystem. So each layer was essentially an ephemeral container. If a build failed, you could actually just run the last successful layer with a shell and see what's wrong.

More importantly, the layers were represented as directories on the host system. So when you wanted to run something in the final container, Docker just needed to reassemble it.

Buildkit has broken all of it. Now building is done, essentially, in a separate system, the "docker buildx" command talks with it over a socket. It transmits the context, and gets the result back as an OCI image that it then needs to unpack.

This is an entirely useless step. It also breaks caching all the time. If you build two images that differ only slightly, the host still gets two full OCI artifacts, even if two containers share most of the layers.

It looks like their Bazel infrastructure optimized it by moving caching down to the file level.

▲

cpuguy83 9 hours ago | parent [-]

Buildkit didn't break anything here except that it each individual build step is no longer exposed as a runnable image in docker. That was unfortunate, but you can actually have buildkit run a command in that filesystem these days, and buildx now even exposes a DAP interface.

Buldkit is far more efficient than the old model.

	▲	cyberax 8 hours ago \| parent [-]
		Buildkit is still a separate system, unlike the old builder. So you get that extra step of importing the result back. And since it's a separate system, there are also these strange limitations. For example, I can't just cache pre-built images in an NFS directory and then just push them into the Buildkit context. There's simply no command for it. Buildkit can only pull them from a registry. > Buldkit is far more efficient than the old model. I've yet to see it work faster than podman+buildah. And it's also just plain buggy. Caching for multi-stage and/or parallel builds has been broken since the beginning. The Docker team just ignores it and closes the bugs: https://github.com/moby/buildkit/issues/1981 https://github.com/moby/buildkit/issues/2274 https://github.com/moby/buildkit/issues/2279 I understand why. I tried to debug it, and simply getting it running under a debugger is an adventure. So far, I found that switching to podman+podman-compose is a better solution. At least my brain is good enough to understand them completely, and contribute fixes if needed.