> avoid a giant prefix miss

I was wondering how feasible it is to modify the context, instead of just appending - e.g. to remove malformed tool calls and their error messages that would pollute the context otherwise.

Sure, you'd have to rebuild the KV cache and take a performance loss, but that might be acceptable if it happens rarely enough.

This is the approach I took. Before leaving scope, results are filtered to remove tool call errors and bloat and instead append those errors to a different system. A much smaller tool call error result is returned to the agent while keeping the reasoning for failure intact.