Remix.run Logo
benswerd 42 minutes ago

Conservatively speaking, LLMs degrade past 40% of a 2M context window, 4k tokens is 0.2%, so no degradation there. Thats also current generation conservative estimate.

I think wasteful is an irrelevant metric. Claude ingests those tokens in a quarter of a second, if it causes it to catch any bug ever it saves far more time than it ever uses.

Any individual default that causes unexpected behavior causes more problems, takes more time and costs more than the small cost of being explicit.