As an SRE I would be very interested on experienced devs point of view on that stance, "we don’t even necessarily need to read the code the LLMs produce". To me, that is the only way a single dev can manage > 1 agent. Because I feel running the code will always be slower than a single agent generating it. On the other hand, it implies lack of human understanding on what is going on under the hood. Which is fine if you trust the LLM to write great code, and great tests for the code, but fundamentally you have to have 100% trust, 99.9% is not going to be enough in any serious industry, would it ?

Also eventually you'll also have to trust it to write the deployment code or even run the deployment itself, otherwise SRE is going to be the bottleneck. And only then should I feel anxiety about the rest of my career (that, or my employer decide LLM are good enough to get rid of me, even if they are imperfect).

We are many of us now engaged in a big experiment to see just how little you can read the code, and for how long. Considering the consensus is that we have still not yet passed the 1 year anniversary of agents getting good, it feels too early to say how it will play out. But there are a lot of market forces working to make it happen.

> Which is fine if you trust the LLM to write great code, and great tests for the code, but fundamentally you have to have 100% trust, 99.9% is not going to be enough in any serious industry, would it ?

This is lacking a lot of nuance. There are many types of code. There are many situations where I'm analysing something one-off and if I get 33% success ratio, but can easily verify the result, I'm happy - still saved me time and money. They're are situations where I'm generating graphs from some dataset and I don't have to trust anything - I know what the result should look like, I just need the agent to drive matplotlib. There are low stakes dashboards which I'm happy to generate and develop entirely via agents - they'll embed the updated screenshots in PRs that I can yolo-merge - worst case is that someone complains about something not working next time they visit. Then there's lots of experimenting which was never stopped to hit production anyway.

Finally after all of that you get code that's actually part of deployable features. Of course the trust is nowhere near 100%, but if you have a healthy testing process (e2e, validating different browsers, or whatever is appropriate for your environment), then what's your trust in human developer+review? Because mine is nowhere near 100% either.

In practice there are places where I extremely don't care about the code and never wanted it anyway, places where I'll read the code to check the design or just in case, and places which agents are not allowed to touch (medical billing rules for example).

Not trying to be a jerk but at this point SRE is my main use case for heavy LLM stuff. It's pretty awesome at that. Deployments and CI pipelines have become a breeze. I used to have to ask DevOps for that and wait days. I have even let agents run deployments for side projects and they seem to do better work than most humans I've worked with. It's wild they will actually read all the logs and debug problems so fast. Not something we could ever keep up with.

The actual code and architecture is where it still lacking IMHO. Especially in rails... Like it will just build the least scalable features if you let it do it's thing. Ten queries for what could be one. No separation of concerns. Huge files, lots of duplicated code and then tens of thousands of units tests that just grow like a fatberg.

If your app does anything serious, if you have serious traffic... you are going to need to review each session finely (and your DB schema with each deploy). It could be that this is maybe an indictment of rails more than LLMs, I guess maybe time will tell.

> Because I feel running the code will always be slower than a single agent generating it.

That is very not true for many cases. Agents generating code are usually painfully slow, finding workflows that replace that reasoning with running code usually speed things up in my experience.