This has been the thing that bugs me. I use LLMs a lot for everything except code that ends up on something I publish, I just keep getting burned enough when I try I stopped. Then people are talking about these 10x gains while the only research we have shows a negative effect.

I don't doubt people are 10x for certain things or things that used to be a ton of plumbing for the smallest stuff, but I'm easily 2-3x faster than I used to be because the rest of the process is so much faster now (finding information, libraries, getting answers to odd questions, builds, BUILDS). But I'm not seeing the software I use or posted here really change or zoom off like people say they are..

It’s selection bias. It’s always been selection bias.

The “10x” developers I’ve known (and been) were all incredibly effective at snagging the work that they personally were most able to do quickly, and shirking the rest.

That’s not necessarily all the difference. But I’d estimate they would maybe be at most 2x developers if you remove their ability to avoid the proverbial 10% of the work that takes 90% of the effort.

AI coding intensifies this effect. And it also helps explain why people’s amazing success stories for personal projects don’t seem to translate to business ROI. For a side project I am in complete control over design and scope, have no customers or stakeholders to keep happy, etc. So I easily - and mostly unconsciously - tailor what I’m doing to what the coding agent can do well, and quietly drop the rest. I don’t have that luxury at work.