Anecdotally (sample size == 1), I have observed that as my context grows longer, the model seems to get dumber. I'm not an advanced user with harnesses, etc.; I just use the chat interface, as that's all I can afford, but after a few turns the model (I'm talking frontier models here from OAI and Ant) seems to just do stupider things.

▲

kadoban 6 hours ago | parent | next [-]

DeepSeek v4.1 flash seems to do this thing where it starts saying that we should wrap up the current work and start again "next session" (whatever that means), that the "context budget" is almost up, or some other wording I'm forgetting, as the context gets longer.

I've taken to just manually making it write a handoff file and clearing context completely. I'm pretty much manually compacting instead of my harness doing it. Feels weird, but it does work.

▲

plasticchris 3 hours ago | parent [-]

I think it’s trained into them so they emit a stop token with enough window left to generate a compaction summary

	▲	kqr 43 minutes ago \| parent \| next [-]
		I would assume it's the harness sending the model a notification toward the end. At least Claude Code is designed to attach reminders with tool call output as a way for the harness itself to steer the model slightly.
	▲	kadoban 3 hours ago \| parent \| prev [-]
		Any idea what the ~rule is or how it's meant to work? I assume it'd work better if I can wire up my harness to handle it, but no idea. It didn't seem to know itself, from it's training. I didn't search too much yet though.

▲

bluerooibos 3 hours ago | parent | prev | next [-]

I can't cite the sources at the moment but this is already well known - a few papers on it and Anthropic say somewhere in their docs that you want to get across the minimum amount of context required to achieve your goal. Quality diminishes the more you dump in.

▲

joshheitzman 6 hours ago | parent | prev | next [-]

Let's bump that sample size to 2. I've observed the same and that observation has heavily influenced the design of my coding agent harness.

▲

jwpapi 6 hours ago | parent [-]

I have auto-compact on at 400k

▲

throw-qqqqq 6 hours ago | parent | next [-]

I try to manually manage as much of the context as possible. Most of the time, I limit myself to a context window of 200k, and it works great for my use cases.

	▲	0xbadcafebee 6 hours ago \| parent [-]
		[dead]

▲

6 hours ago | parent | prev [-]

[deleted]

▲

viccis 2 hours ago | parent | prev [-]

It does, and, when combined with Claude's stingy cache TTL of 5 minutes (unless you up it to 1h at a sizeable increase in token cost) for API billing, means that some sessions cost like 10x what they should because of cold caches.