The problem is, when the context window grows, Claude tends to forget these kinds of rules. It will then do whatever it wants. I had to outright ban comments in the global claude.md, the local claude.md AND write a hook to catch any that still slipped through.

▲

nater5000 7 hours ago | parent | next [-]

I think people really need to focus more on working with limited contexts rather than trying to work around it. I really try to keep my sessions as short as possible and it helps a ton with keeping Claude (et al) focused.

Specifically, I like the "canary" trick that people have discussed where you add a small, innocuous rule to your CLAUDE.md like "When responding to me, start every sentence with my name." so that when Claude stops doing this, you know you've used way too much context and need to start a new session.

▲

faizshah 6 hours ago | parent [-]

This or you just repeat the initial prompt every 200k tokens

	▲	ianjbutler 3 hours ago \| parent [-]
		Which gets you to the point where the whole thing is.. still unreliable. Generative text engines are going to generate. This calls for real enforcement in deterministic pre-edit hooks. And here is where naive people will say something like "Why do I care if robots shit all over the codebase? Code is for machines, I don't expect to deal with it much now". But really externalized CoT like this confuses machines too, wastes tokens, and eventually wastes exponentially many tokens. Agents tend to think it's more real grounding than prompts are, even for comments-in-code. One bad comment poisons everything, then gets copied around as a ground-truth assumption everywhere. Hooks are more real to them than prompts or comments, and even then if you add enforced limits and tell them to externalize CoT ONLY in scratch task-tracking docs.. they will violate comment-enforcement hooks about 25% of the time. That tells you everything you need to know: even with constant reinforcement, they just really want to break this kind of rule.

▲

hectdev 24 minutes ago | parent | prev | next [-]

Yea, I've started making it write linters to check the code that goes out. Anything that can be deterministically measured, gets added to it once we lock it down.

▲

mandeepj 7 hours ago | parent | prev | next [-]

> The problem is, when the context window grows,

You know the problem; then why not address it? Does Compacting the context not help?

▲

adastra22 6 hours ago | parent | next [-]

Compacting the conversation almost never helps. It is uniformly worse than starting over with fresh context, or rewinding to a last-known-good state. It only exists because it increases engagement.

	▲	enraged_camel 2 hours ago \| parent [-]
		This does not match my experience. I use long-running orchestrator sessions. Each orchestrator is in charge of planning, writing kick-off prompts for implementers, answering questions from those implementers, doing code reviews and providing feedback, and answering side questions from me when I have them. Depending on the initiative I might compact a session a dozen times, sometimes more. It is lossy, and the session certainly tends to forget earlier bits as more compactions happen, but overall it's a much better experience than starting fresh and having to re-explain everything. The only time I compact is if the session goes wildly off-course and the context gets polluted with off-topic conversations. Also worth noting: with Claude Code you can provide custom instructions when compacting, and instruct the LLM that is in charge of compacting the session to prioritize the retention of specific bits. It can help a lot.

▲

cautiouscat 6 hours ago | parent | prev | next [-]

Compaction is a main cause of this problem.

▲

troupo 7 hours ago | parent | prev [-]

Compacting context compacts context. So Claude forgets a lot during compaction.

▲

Maxatar 6 hours ago | parent [-]

Compacting mostly gets rid of reasoning tokens, and honestly it would be nice of reasoning tokens did not constantly follow every follow up query. Asking even a simple/trivial question can have Claude use thousands of tokens. Compacting is good for getting rid of those.

▲

troupo 6 hours ago | parent [-]

I've had Claude immediately fall back to its usual verbose style immediately after compaction.

To be fair, I've had it do that immediately after re-reading the output style instructions, too.

My chat history is filled with "Yes, I broke the language rule. Let me rephrase that and update my memory. — You already have that in memory — Yes, true, I ignored that" (because "Memory" is a yet another .md file)

▲

strbean 3 hours ago | parent [-]

Claude Code supposedly supports a "post-compaction" hook, so you could have it automatically run the prompt "We just compacted the context, quickly refresh yourself on the rules in CLAUDE.md etc..".

Depending on what you've got in those files, maybe that will just use up all the context again though.

▲

troupo 2 hours ago | parent [-]

> supposedly supports a "post-compaction" hook, so you could have it automatically run the prompt

Keyword "supposedly" :)

I've had it in my settings forever, and still...

Asking it to analyse and fix the issue it produced a plausible "my training supercedes/overrides settings especially if triggered by certain words in the phrase" (paraphrasing the long text)

	▲	strbean 2 hours ago \| parent [-]
		> Keyword "supposedly" :) > I've had it in my settings forever, and still... Checks out! I've never used it my self, so it I figured it likely didn't work at all.

▲

aleksiy123 3 hours ago | parent | prev [-]

Hooks is the way.

Intermittent nudges