Naive junior engineers eventually learn and become competent senior engineers. LLMs forget everything they "learn" as soon as the context window gets too big.

Very true! I liken AI to having an endless supply of newly hired interns with near-infinite knowledge but intern-level skills.

There are like a dozen well-established ways to overcome this. Learn how to use the basic tools and patterns my dude.