> LLMs already write code better than most humans.

If you mean better than most humans considering the set of all humans, sure. But they write code worse than most humans who have learned how to write code. That's not very promising for them developing intelligence.

I have been writing code for nearly 30 years. I have my name on a couple of ingenious patents.

I have fully integrated LLMs into May daily workflow, and am routinely surprised at the innovative solutions that Claude sometimes comes up with. There is a lot of generated code that is far beyond my own capabilities, even as an expert in the field in which I’m working.