The problem is it’s impossibly hard to test all the edge cases

Which is probably why so many random buttons in microsoft/apple/spotify just stop working once you get off the beaten path or load the app in some state which is slightly off base

Easy, have Claude review the code, tell it to be critical and that it needs to be easier to understand, follow Clean Code, SOLID principles and best practices. Lie to it, say you got this from a Junior developer, or "review it as if you were a Staff Level Engineer reviewing Junior code" the models can write better code, just nobody tells them to.

I keep hearing the assertion that you can’t make high quality, maintainable code with LLMs. The last two years using AI have shown me exactly the opposite.

I think it’s all about the structure you use to work in and how you use the model. We are shipping better, more human friendly code, with less bugs, then we ever did before and doing it at 1/10 the cost before LLMs.

But we are definitely not vibe coding, and the key seems to be devs with years of experience managing teams, managing the LLM instead. Basically you create the same kind of formal specifications, conventions, and documentation that you would develop for a project with two or three teams, then use that to keep the project on the rails recursively looping back through the docs as you go along. I’ve only had to back out of a couple of issues over the last year, and even though that cost a couple of hours, it was still extremely cheap.

Meanwhile we are shipping at 4x speed with 1/4 the labor, and the code is better than it was because the “overhead” of writing maintainable, self documented code has inverted into the secret ingredient to shipping bug free code at unprecedented speed.

If you just explain the standards to which you want the code written, use a strict style guide, have a separate process that ensures test coverage (not in the same context) you can get example quality code all the way through. Turns out that’s also in the training data.

> the clean modeled structure we want to see is for humans to maintain and extend, but the AI doesn't need this.

This does not match my experience. I do a lot of AI-assisted coding at this point, and what I've seen is that when the AI is asked to extend or modify existing code, it does a much better job on clean, well-structured and well-abstracted code.

I think the reason is simple, and tracks for humans as well: well-structured code is simply easier to understand and reason about, and takes a smaller amount of working-set memory. Even as LLMs get better with coding, I expect that they would converge on the same conclusion, namely that good structure + good abstractions make for code that is more efficient to work with.

The reason why you will never get software engineers (in companies) to accept the man behind the curtain is liability. If a human software engineer is still responsible for what happens when the AI developed code has a catastrophic bug or security vulnerability, then the only way for the human to know if there is a problem is to be able to read through the code or run it through some <insert advanced formal verification tool here> that guarantees zero issues.

I think we eventually end up at the tool approach via vendors providing the tools to other companies, but it still feels like there's a long road ahead to get there.

Many of us recognize that the days of nearly-free tokens is quickly drawing to a close, and at some point humans may very well have to dig their keyboards out of cold storage and return once again to the code mines.

> but the AI doesn't need this

That's not true. The LLM performance will degrade as the codebase gets messier as well. You get to a point where every fix breaks something else and you can't really make forward progress.

Yes, you might be able to get a bit further with a messy codebase just because the LLM won't complain and will just grind through fixing things, but eventually it will just start disabling failing tests instead of actually fixing things.

Sometimes I think the main value in AI-maintained code being “high quality” is when the structure can enforce invariants. If invalid states aren’t representable, then the AI can’t easily add bugs in the future.

Of course that just leads to: what’s the best way to achieve that goal: through elegant code or adding lots of tests? Which is a debate from long before LLMs existed.

>This just might be the new reality, and we need to stop looking behind the curtain and accept what the wizard presents us.

This is how societies become shittier. People who are ostensibly responsible for doing their jobs not giving a damn about quality.

> Why was it a maintenance dead end?

LLMs have a limit to how deep they can understand and refactor architectural issues.

That limit is far, far lower than a human's.

Yeah that’s my main concern. It feels so so easy to be lazy and do a bad job now. And then my skills weaken and what makes me valuable fades.

I love the Iron lung reference. Perfect.

At that point the code becomes a compile target, and then you need a new source of truth.

Which I think is perfectly worthy of exploration. Some people want to check in the prompts. Or even better, check in a plan.md or evenest betterest: some set of very well-defined specifications.

I'm not sure what the answer will be. Probably some mix of things. But today it is absolutely imperative that the code I write for the case I wrote it in is good quality and can be maintained by more than just me.

Same. Messy code makes it harder for us to understand and thus maintain the code (which is why people often refer to code as a liability), but is that the case for AI tools as well? If not, it seems like clean code may not matter as much anymore.

The problem with crappy frontend code is not only the maintenance. It's that stuff such as responsive design, accessibility or cross-browser compatibility that work nearly for free with elegant code won't work at all.

The problem is that technical debt is compounding. Bad LLM architectural and implementation decisions just blend in to the background and you build layer upon layer of a mess. At some point it becomes difficult and expensive (token wise) to maintain this code, even for an agent.

I mitigate this by few things: 1. Checkpoints every few days to thoroughly review and flag issues. Asking the LLM to impersonate (Linus Torvalds is my favorite) yields different results. 2. Frequent refactors. LLMs don't get discouraged from throwing things out like humans do. So I ask for a refactor when enough stuff accumulates. 3. Use verbose, typed languages. C# on the backend, TypeScript on the frontend.

Does it produce quality code? Locally yes, architecturally I don't know - it works so far, I guess. Anyway, my alternative is not to make this software I'm writing better but not making it at all for the lack of time, so even if it's subpar it still brings business value.

He was presumably also not constructing a powered exoskeleton of from fictional materials or a physically implausible power source, but since you obviously caught the reference, how about some benevolent interpretation instead, for a decent shorthand about working smoothly with AI assistance.

(And on a personal note, I'm glad we don't have a publicly released Jarvis before we get our act together about the use.)