Prefix caching is inherent in how global softmax attention works. Modifying the prefix invalidates the suffix because the suffix's cached KV projections are a function of the hidden state after the model has evaluated the prefix.

There have been some attempts to approximately stitch KV blocks without full re-evaluation (see: CacheBlend on arxiv), but the results aren't promising.

You mentioned HiCache: as far as I can tell this is a prefix tree cache that efficiently shares prefixes across related histories. If you change the prefix, everything after the change still misses the cache and needs to be prefilled again.