And there is another problem: LLMs generating too much code, code that is doing more than was asked. And that cannot be fixed by tests. Usually, we create tests for wanted behavior and expected exceptions. But we don't create tests for undesired behavior.

▲

flyinglizard 3 days ago | parent | next [-]

Code cost is almost down to zero. If you move the point of “just leave it to the machine” from the compiler (where humans used to do the coding) to the high level logic (now with LLMs) then in most cases more code does not really matter. Like, why build and maintain an abstraction where the LLM could implement this many times over each time with different subtleties? Why use a library with its own constraints when you could have exactly what you want? Why use cross platform frameworks when you can just one shot the thing to N different platforms? It’s not even slower. You can have code that’s larger yet more performant (stripping away abstractions can do that).

From time to time I try to do a pass of coalescing flows and cases and removing dead code to reduce the context and prevent the LLM from tripping over itself. But if it’s exclusively LLM maintained code I don’t care too much if there’s more of it.

	▲	pdimitar 3 days ago \| parent [-]
		The "exactly" part is the problem. Just last two weeks I had to slap Fable, three times, to stop writing 1000-2000 lines of defensive code... because of DB columns I just forgot should be NOT NULL. That was it. Nothing else. I told it that, boom, -4800 coding lines: gone. LLMs defend the status quo and they regularly lose sight of everything bigger than the current PR they are working on. I too am gradually making peace with the fact that LLM-maintained code does not have to be 100% readable for humans. But this is not about readability. It's about the data model. So one concession I am willing to make is: don't care too much about the code _BUT_ manually curate the data model. So far: small wins on iteration turns and code volume producing. Too early to tell but for now I am happy with the results.

▲

user43928 3 days ago | parent | prev [-]

Have you worked with Opus 5?

Its documentation about what the code does not do could fill whole books.

UI copy being full of slop explaining what the software does not do is another problem.

I am not convinced that a lack of negative test cases is an issue.

I do agree it generates too much code most of the time.

	▲	lukeschlather 3 days ago \| parent [-]
		Documentation is important. I would say Opus' propensity to write documentation that documents non-features is part of the problem being discussed. And the problem isn't just that it says what the software doesn't do, most of the things it claims are in fact meaningless, it's not even clearly describing something the software shouldn't do.