Remix.run Logo
storus 3 hours ago

Can it instruct DeepSeek during an LLM call to start removing old tool calls from the context instead of waiting for the LLM call to finish if the context size approaches DeepSeek's dumb zone? Claude Code can't do that, /compact can only happen after the LLM call; it's often preferable to start cleaning up context during an LLM call, especially when tool calls are huge like reading markdown files; implementation-wise all that is needed is to start removing earliest <tool call start> ... <tool call end> and replacing them just with some log entry stating this tool call was already performed, then re-running KV cache prefill (so the "online" compaction would get 0.5s latency hit every time it's performed). That way one can read 1000 files in one LLM call.