That'd be really easy to spot and also fix, most likely. Any open issue you could point us to, must surely been reported already?

▲

nolok 5 hours ago | parent | next [-]

> That'd be really easy to spot and also fix, most likely

Ah, reminds me of good old "There are only 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors."

▲

criemen 4 hours ago | parent [-]

> Ah, reminds me of good old "There are only 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors."

You quip, but LLM KV caching (from the harness side) is quite easy: You get a cache hit on stable prompt prefixes, period. That means you want to keep the prefix stable, and only append at the end of the conversation. Made up example: Don't put the git branch name into the system prompt part (that comes first), as whenever the branch name changes, that'd trigger a cache invalidation of the entire prompt.

Getting this right requires some care to not by accident modify the prefix, basically, and some design on communicating the things that can change (user configuration, working dir, git information, ...).

▲

franknord23 3 hours ago | parent | next [-]

That sounds like the experience of writing Containerfiles; since steps are cached you want to pull the thing you are iterating on as far down as possible.

	▲	gopher_space an hour ago \| parent [-]
		All of this work has been done before in different contexts. Memory management with bigger blocks and weaker definitions that change whenever some grad student gets a bright idea.

▲

xcjsam 4 hours ago | parent | prev [-]

[dead]

▲

krzyk 5 hours ago | parent | prev [-]

Opencode (and other coding agents) have hundreds of open issues reported. It is quite discouraging when they are not being closed/fixed.

	▲	jsjsjsuduiwkw 4 hours ago \| parent [-]
		[dead]