Buildkit can be very efficient at caching, but you need to design your image build around it. Once any step encounters a cache miss, all remaining steps will too.

I'd also avoid loading the result back into the docker daemon unless you really need it there. Buildkit can output directly to a registry, or an OCI Layout, each of which will maintain the image digest and support multi-platform images (admittedly, those problems go away with the containerd storage changes happening, but it's still an additional export/import that can be skipped).

All that said, I think caching is often the wrong goal. Personally, I want reproducible builds, and those should bypass any cache to verify each step always has the same output. Also, when saving the cache, every build caches every step, even if they aren't used in future builds. As a result, for my own projects, the net result of adding a cache could be slower builds.

Instead of catching the image build steps, I think where we should be spending a lot more effort is in creating local proxies of upstream dependencies, removing the network overhead of pulling dependencies on every build. Compute intensive build steps would still be slow, but a significant number of image builds could be sped up with a proxy at the CI server level without tuning builds individually.

> Buildkit can be very efficient at caching, but you need to design your image build around it.

Well, that's what I've been trying to do. And failing, because it simply doesn't work.

> I'd also avoid loading the result back into the docker daemon unless you really need it there.

I need Docker to provide me a reproducible environment to run lints, inspections, UI tests and so on. These images are quite massive. And because caching in Docker is broken, they were getting rebuilt every time we did a push.

Well. I switched to Podman and podman-compose. Now they do get cached, and the build time is within ~1 min with the help of the GHA cache.

And yes, my deployment builds are produced without any caching.