> If you're completely new to the problem then ... yes, it does.

Of course, because I am not new to the problem, whereas an LLM is new to it every new prompt. I am not really trying to find a fair comparison because I believe humans have an unfair advantage in this instance, and am trying to make that point, rather than compare like for like abilities. I think we'll find even with all the context clues from MCPs and history etc. they might still fail to have the insight to recall the right data into the context, but that's just a feeling I have from working with Claude Code for a while. Because I instruct it to do those things, like look through git log, check the documentation etc, and it sometimes finds a path through to an insight but it's just as likely to get lost.

I alluded to it somewhere else but my experience with massive context windows so far has just been that it distracts the LLM. We are usually guiding it down a path with each new prompt and have a specific subset of information to give it, and so pumping the context full of unrelated code at the start seems to derail it from that path. That's anecdotal, though I encourage you to try messing around with it.

As always, there's a good chance I will eat my hat some day.

I call this context decay. :)

The bigger the context, the more stuff "decays" sometimes to complete different meanings

The brain can literally not process any piece of information without being changed by the act of processing it. Neuronal pathways are constantly being reinforced or weakened.

Even remembering alters the memory being recalled, entirely unlike how computers work.

I'm not sure the idea that a developer maintains a high level understanding is all that controversial...

I never intended to say it was true of all codebases for all developers, that would make no sense. I don't know all developers.

I think it's objectively true that the information is not in the LLM. It did not have all codebases to train with, and they do not (immediately) retrain on the codebases they encounter through usage.

And when you work on a really big codebase you start having multiple files and have to learn tools more advanced than ctrl-f!!

We're measuring lengths of string, but I would not say I have worked on small projects. I am very familiar with discovery, and have worked on a lot of large legacy projects that have no tests just fine.

Why are LLMs so bad at doing the same thing?

Of course, but even those blackoxes are not empty, they've got a vague picture inside them based on prior experience. I have been doing this for a while so most things are just various flavours of the same stuff, especially in enterprise software.

The important thing in this context is that I know it's all there, I don't have to grep the codebase to fill up my context, and my understanding of the holistic project does not change each time I am booted up.

And LLMs can’t leverage these abstractions nearly as well as humans…so far.

We don't actually know that.

If we had that level of understanding of how exactly our brains do what they do things would be quite different.

The comparison would be apt if the LLM was trained on your codebase.

You have no idea if I remember with pixel perfect accuracy (whatever that even means). There are plenty of people with photographic memory.

Also, you're a programmer you have no foundation of knowledge on which to make that assessment. You might as well opine on quarks or martian cellular life. My god the arrogance of people in my industry.

This is a thing and doesn't require a separate model. You can set up custom prompts that will, based on another prompt describing the task to achieve, generate information about the codebase and a set of TODOs to accomplish the task, generating markdown files with a summarized version of the relevant knowledge and prompting you again to refine that summary if needed. You can then use these files to let the agent take over without going on a wild goose chase.

frontend path: /src/frontend/* backend path: /src/*

I suppose the problem you have might be unique to nextJS ?

That's a really anthropomorphizing description; a more mechanical one might be,

The attention mechanism that transformers use to find information in the context is, in its simplest form, O(n^2); for each token position, the model considers whether relevant information has been produced at the position of every other token.

To preserve performance when really long contexts are used, current-generation LLMs use various ways to consider fewer positions in the context; for example, they might only consider the 4096 "most likely" places to matter (de-emphasizing large numbers of "subtle hints" that something isn't correct), or they might have some way of combining multiple tokens worth of information into a single value (losing some fine detail).

> I think you misunderstand how context in current LLMs works.

Thanks but I don't and I'm not sure why you're jumping to this conclusion.

EDIT: Oh I think you're talking about the last bit of the comment! If you read the one before I say that feeding it the entire repo isn't a great idea. But great idea or not, people want to do it, and it illustrates that as context window increases it creates demand for even larger context windows.

It seems like LLM need to become experts at managing their OWN context.

Selectively gripping and searching the code to pull into context only those parts relevant to the task at hand.