> and so has the output quality

Not my experience at all. The quality has gone up dramatically. Because we also use LLMs to make large scale refactorings that cleans up the code and use it to find lots of bugs that would otherwise have exploded in production at some point in time.

I just used Fable to thoroughly review a PR, and it came up with 5 "Findings that block approval".

I went through each one through the code path (a couple were already considered during implementation), and while all 5 were theoretically accurate, 0 of them were actually reachable.

I wrote them back to Claude point by point and got this back: "I checked all five claims against the code. Your list is correct on all five. I downgrade my verdict: none of the five findings blocks approval. Details and one subtlety below."

This is a usual experience with SOTA models for me and others in my team.

Don't get me wrong, I use it daily; sometimes as a crutch, other times as an accelerator, mostly as a validator/peace of mind. I however don't think that neither velocity or quality is meaningfully changed. But I do spend much more time arguing with a machine.