Yeah, I've experienced similar stuff. Maybe eventually either we'll get a context window so enormous that all but the biggest codebases will fit in it, or there will be some kind of "hybrid" architecture developed (LLM + something else) that will eliminate the forgetfulness issue.