Good diagnosis of the loop. But “LLMs can’t build software” is mostly a statement about interface, not capability. If we ask a stochastic parrot to behave like a staff engineer in its head, it fails. If we reshape the work so the mental model lives outside the model—executable specs, tight tests, ADRs as shared state, small DSLs, and an orchestrator that forces evidence before code—agents can move through the same loop on bounded problems. In other words, they don’t need an internal model if the environment gives them one.

The better question isn’t “can an LLM maintain two mental models?” but “how much of this problem can we make machine-checkable?” Where we can’t (socio-technical trade-offs, ambiguous requirements), a human owns the decisions. Where we can (migrations, glue, refactors guarded by tests), the agent owns the keystrokes.

Today’s failure modes (omission, recency bias, hallucination) are real, but mitigated by durable memory, runbooks, and mandated check-ins the tool can’t skip. So: not “can’t build software”, but “can’t be the tech lead”. Yet.