> I don't see how that's possible if the cache no longer stores the uncompressed data.

Zstd has a seekable format for frames, similar to pigz --independent works.

[1] - https://github.com/facebook/zstd/blob/dev/contrib/seekable_f...