Because the exact same processes that create it faster, are also now generating large comprehensive test suites, and data-validation suites based around the actual business requirements and data/edge cases.

In essence, it is writing the code faster (great), but then also writing the code that was often previously never written due to budget/time limitations (comprehensive testing/validation).

So, how do you know the code is the highest quality ever? Because we have test result data that PROVES it to be the case. It isn't "trust but verify" it is "Don't trust. Verify."

This is my observation as well. I thought I was good at testing (I wanted to be good at testing), but the models are just better at it. Testing in particular rewards a tireless cataloging of the tiny and the tedious and in this respect LLMs are unmatched.

Single data point, my Sentry notifications across more than a half dozen client projects when from 2-4 a month to basically 0. That's while shipping 20x _more_ code.

Yeah but you don't test for stuff you don't know you have to test for. All tests can pass and internally its still done in a braindead way that will explode at some point.

How do you know those large comprehensive test and validation suites you've never looked at actually reflect real user experiences at all?

> Because we have test result data that PROVES it to be the case.

pgrust thought this too, but then another human discovered claude had hardcoded benchmark query plans.

Now pgrust has a "test" [1] to make sure the Ai doesn't do this again (unless it decides this test is bad for the current goal)

I have a growing list of these, another notable one is OpenCode having a security regression [2] because the Ai redesigned configuration loading for v2.

Real verification seems beyond transformer capabilities.

[1] https://github.com/malisper/pgrust/blob/38634b408dea373f3944...

[2] https://github.com/anomalyco/opencode/pull/54921