Remix.run Logo
maccard 3 hours ago

> I also think I write better code than an LLM in certain areas, but in most programming domains, the LLM knows more than I do across many dimensions.

Remember that the quality of the LLMs code in the areas you don’t know is as good as the quality as the area you do know. You’re only able to gauge the quality of what you do know.

jdw64 3 hours ago | parent | next [-]

Your main target seems to be the view that software quality is a matter of 'aesthetic intuition,' but I see it differently. I think because it's 'engineering,' there must be measurable indicators.

Executable specifications

Unit, integration, and property tests

Official API documentation

All of these provide ways to judge quality. There are so many metrics that the problem is actually choosing which ones to use.

1.Does it produce output A for input B? 2.Can it process 100,000 records within 5ms? 3.Is memory usage within the defined limit? 4.Does the protocol handle error conditions properly?

You don't need to be an expert to test these. People forget that programming is one of the few fields where judgment criteria can be easily translated into machine executable tests.

The biggest problem with epistemological objections is that they often assume a binary: 'experts can judge, non-experts cannot.' But in reality, it's a continuous process.

You run the code, notice something odd, look up the terminology, compare it with official documentation and reference implementations, add tests, and gradually build up judgment in that area. Rewriting LLM generated code compresses that entire learning curve.

In the past, entering an unfamiliar domain meant starting from a blank file and reading documentation. Now, you start with a working hypothesis and modify it. That's far cheaper than starting from scratch.

If you treat LLM generated code as executable teaching material and a falsifiable hypothesis, it's a very powerful resource.

I feel uncomfortable when people reduce programming to aesthetic quality alone. We were all trained to measure things.

TheOtherHobbes 2 hours ago | parent [-]

My "aesthetic" sense is more about how modular the code is, how DRY (which is also an aesthetic balancing act), how clean the abstractions are and how well they fit the domain and the workflow.

Also, error handling.

Tests, specs, and docs are are all downstream of that.

So far I've found that AI does an adequate-to-very-good job up to a certain amount of code, then things tend to fall apart. The solution is modularity and clean interfaces - as it always was.

jdw64 2 hours ago | parent [-]

I'm not really sure. I'm mostly thrown into existing codebases and just modify things to fit the existing style, so I'm almost always evaluated purely by metrics. I do think your (TheOtherHobbes) standards are really good, though.

jdw64 3 hours ago | parent | prev [-]

>You’re only able to gauge the quality of what you do know.

That's not true. You can follow the logic step by step, search for similar domains, and form a comparison group.

Realistically, if we take this logic to its conclusion, it's like asking: 'You're just a spectator, so can you really judge whether a football player is good at football?

Realistically, LLMs write code well. I'm a C# programmer, and as you know, the current trend in modern programming is to refine error handling—using try/catch at higher layers, working with monads, preserving computable contexts, and branching accordingly. That kind of code typically takes 1 to 2 hours of modeling before you even start writing.

But an LLM can do it in just 5 minutes.

So I think the opposite way: if LLMs can do this well in a domain I'm familiar with, they're probably just as capable in other domains too. And that makes them feel trustworthy.

And more importantly, no matter how great a human programmer is, once you step outside your domain, you're in unknown territory. In those areas, LLMs are surprisingly helpful.

Epistemologically, it might be true that you can't evaluate code in a domain you don't know. But people keep forgetting the fact that you can learn through that code. This very post is itself an example of learning through LLM-typed code.

And programming actually provides clearer results than many other domains. It's not based on subjective UX like UI design. You have:

1.Executable specifications

2.Performance measurements

3.Comparisons with official documentation

These are clear engineering outcomes that allow for objective comparison. For example, you can verify whether a program produces results within 5ms when executed.

Many epistemological arguments tend to overlook this.

A lot of code is open source. You can find mature implementations anywhere. Finding a repo from someone claiming 10 or 20 years of experience and running a differential comparison is not difficult at all."