I have not seen evidence that they are regression to the mean machines.

I'm lucky to work with great engineers and their productivity and code quality has become even higher. Wish that wasn't the case, but it is, and that puts also lots of pressure on myself to work more and better all the time. It's exhausting.

There are cons too, system's understanding sometimes is not as intimate, which in turn produces less "gotcha" moments that may lead to better design. There's less time to review PRs and make it a choral work.

On the other hand way more refactors and experiments can be run, so again, code quality has improved just because if you have a hunch that something could be done better, you can test it for cheap.

▲

canadaduane 2 hours ago | parent [-]

I'm curious what you think of as "the mean"? I consider the input training set for an LLM to contain its mean. My hypothesis would be: an LLM alone cannot consistently produce code above the mean of the quality it was trained on.

	▲	epolanski an hour ago \| parent [-]
		The input training doesn't matter much, besides, the input training is already skewed for code that has been submitted after much trial and error by a dev locally and possibly reviewed. And input has an over bias over open source projects, not crap internal tools no llm has ever seen. There's more to the quality of the output, like prompts, the quality of the codebase (from which the llms learn), the documentation/harnessing, the feedback an engineer provides while reviewing multiple times (in the chat, in the diff, in the pr) etc, etc.