Yes. It’s perfectly reasonable to expect the user to know the intricacies of the caching strategy of their llm. Totally reasonable expectation.

▲

jghn 4 hours ago | parent | next [-]

To some extent I'd say it is indeed reasonable. I had observed the effect for a while: if I walked away from a session I noticed that my next prompt would chew up a bunch of context. And that led me to do some digging, at which point I discovered their prompt caching.

So while I'd agree with your sarcasm that expecting users to be experts of the system is a big ask, where I disagree with you is that users should be curious and actively attempting to understand how it works around them. Given that the tooling changes often, this is an endless job.

▲

abustamam 4 hours ago | parent [-]

> users should be curious and actively attempting to understand how it works

Have you ever talked with users?

> this is an endless job

Indeed. If we spend all our time learning what changed with all our tooling when it changes without proper documentation then we spend all our working lives keeping up instead of doing our actual jobs.

	▲	Octoth0rpe 2 hours ago \| parent [-]
		There are general users of the average SaaS, and there are claude code users. There's no doubt in my mind that our expectations should be somewhat higher for CC users re: memory. I'm personally not completely convinced that cache eviction should be part of their thought process while using CC, but it's not _that_ much of a stretch.

▲

coldtea 6 hours ago | parent | prev [-]

It's not like they have a poweful all-knowing oracle that can explain it to them at their dispos... oh, wait!

▲

esafak 6 hours ago | parent [-]

They have to know that this could bite them and to ask the question first.

▲

nixpulvis 5 hours ago | parent [-]

I do think having some insight into the current state of the cache and a realistic estimate for prompt token use is something we should demand.

	▲	switchbak an hour ago \| parent [-]
		If there was an affordance on the TUI that made this visible and encouraged users to learn more - that would go a long way.