interesting take. I think I disagree, but I like this take a lot and I had to think about it.

First, I think that models still need a context layer. One way to think about 'context' is as a form of compression. You provide the model context because it makes it easier for the model to figure out what to do. Even in a world with infinite model capacity and infinite model context, this is still useful because it allows the model to avoid rederiving everything from first principles every time. As long as models perform better using fewer tokens and as long as we care about token spend, context is a useful (necessary?) shortcut.

Once you bite that you need some form of context layer, the question is which. Here I do agree that it is better to work with what the models will find familiar (markdown files colocated with code, for eg). But this speaks to over-engineered solutions not understanding their main user (the agent) more than it does the need or lack there of.

A) Context and prompting cuts the search space for next token generation. That’s pretty useful, as you mentioned.

B) The other use of context is that it introduces entirely new information via RAG

B will never go away (as others pointed out). A, well that’s just something we’re all going to keep getting surprised at. We’ll barely give it any direction or context and the newer models will simply find the happy path.

The author is kind of suggesting that their context wasn’t really necessary to get the happy output, I think.

Chain of reasoning is a lot of context to guide token generation, but we simply see that newer models don’t need that context to get to the answer. I’m mostly reiterating this because there’s a hot take here, and that is this agentic stuff may be waived away by magic frontier-llm wand , all of a sudden.