I believe issues will arive with AI in the near future in terms of their performance. Disagreeing with the author, I think code quality pre-AI, in certain languages, were golden. They all contained bugs but they were written in a way that bug sources would be more or less obvious and most but not all top open source software had linting guidelines, code styles & friends to help make it ingestible. Perhaps these helped said project's maintainers a great lot, but it absolutely SAVED AI when it came to training data. These top-quality repositories with thousands of lines of great code combined with hundred thousand lines of mixed quality code was the perfect formula for the big data churner™. Now what? All those repos, inflated with stars are generated by one of the 5 LLMs out there. There has been extended discussions about deterministic behaviour in LLMs, and I'm no ML engineer but to me this drop in entropy will surely cause backtracking in code quality. Of course there a vast array of improvements that can be made outside of training data, but the whole psychology attached to LLM marketing, in my opinion, obstructs those improvements. You could train a better suited way of input for LLMs, set cutoffs and posttraining in just the right places and everything but you gotta think it through. How many of those who ask their favourite LLM everything think anymore? How much of our current knowledge is safe from AI hallucinations or subtle nudges? AI has changed the world of coding, but the current state in Anthropic HQ will determine if it will just be mid-quality codegen or accelerationist fever dream.

I think it's related that we as humans see when something becomes hard to reason about, and decide to refactor it. I'm not sure whether an LLM with full ownership of a codebase could do that, for its own benefit.

I do see a world where models could be trained on it, but I imagine it will be more expensive, because it requires including future rewards about the models' own later efficiency.