Remix.run Logo
jdw64 3 hours ago

I think there are far more people who can't write better code than an LLM. Of course, there are a few exceptions, but it's a fact that LLMs are already handling PhD-level mathematics and papers.

I also think I write better code than an LLM in certain areas, but in most programming domains, the LLM knows more than I do across many dimensions. As prompts get deeper, LLMs are already producing PhD-level code—and that's been shown in research. The vast majority of people don't have that level of education. Of course, having a PhD doesn't guarantee good coding, but at least it's clear that LLMs can handle that level of capability.

People might disagree, but my view is firm on this.

maccard 3 hours ago | parent | next [-]

> I also think I write better code than an LLM in certain areas, but in most programming domains, the LLM knows more than I do across many dimensions.

Remember that the quality of the LLMs code in the areas you don’t know is as good as the quality as the area you do know. You’re only able to gauge the quality of what you do know.

jdw64 3 hours ago | parent | next [-]

Your main target seems to be the view that software quality is a matter of 'aesthetic intuition,' but I see it differently. I think because it's 'engineering,' there must be measurable indicators.

Executable specifications

Unit, integration, and property tests

Official API documentation

All of these provide ways to judge quality. There are so many metrics that the problem is actually choosing which ones to use.

1.Does it produce output A for input B? 2.Can it process 100,000 records within 5ms? 3.Is memory usage within the defined limit? 4.Does the protocol handle error conditions properly?

You don't need to be an expert to test these. People forget that programming is one of the few fields where judgment criteria can be easily translated into machine executable tests.

The biggest problem with epistemological objections is that they often assume a binary: 'experts can judge, non-experts cannot.' But in reality, it's a continuous process.

You run the code, notice something odd, look up the terminology, compare it with official documentation and reference implementations, add tests, and gradually build up judgment in that area. Rewriting LLM generated code compresses that entire learning curve.

In the past, entering an unfamiliar domain meant starting from a blank file and reading documentation. Now, you start with a working hypothesis and modify it. That's far cheaper than starting from scratch.

If you treat LLM generated code as executable teaching material and a falsifiable hypothesis, it's a very powerful resource.

I feel uncomfortable when people reduce programming to aesthetic quality alone. We were all trained to measure things.

TheOtherHobbes 2 hours ago | parent [-]

My "aesthetic" sense is more about how modular the code is, how DRY (which is also an aesthetic balancing act), how clean the abstractions are and how well they fit the domain and the workflow.

Also, error handling.

Tests, specs, and docs are are all downstream of that.

So far I've found that AI does an adequate-to-very-good job up to a certain amount of code, then things tend to fall apart. The solution is modularity and clean interfaces - as it always was.

jdw64 2 hours ago | parent [-]

I'm not really sure. I'm mostly thrown into existing codebases and just modify things to fit the existing style, so I'm almost always evaluated purely by metrics. I do think your (TheOtherHobbes) standards are really good, though.

jdw64 3 hours ago | parent | prev [-]

>You’re only able to gauge the quality of what you do know.

That's not true. You can follow the logic step by step, search for similar domains, and form a comparison group.

Realistically, if we take this logic to its conclusion, it's like asking: 'You're just a spectator, so can you really judge whether a football player is good at football?

Realistically, LLMs write code well. I'm a C# programmer, and as you know, the current trend in modern programming is to refine error handling—using try/catch at higher layers, working with monads, preserving computable contexts, and branching accordingly. That kind of code typically takes 1 to 2 hours of modeling before you even start writing.

But an LLM can do it in just 5 minutes.

So I think the opposite way: if LLMs can do this well in a domain I'm familiar with, they're probably just as capable in other domains too. And that makes them feel trustworthy.

And more importantly, no matter how great a human programmer is, once you step outside your domain, you're in unknown territory. In those areas, LLMs are surprisingly helpful.

Epistemologically, it might be true that you can't evaluate code in a domain you don't know. But people keep forgetting the fact that you can learn through that code. This very post is itself an example of learning through LLM-typed code.

And programming actually provides clearer results than many other domains. It's not based on subjective UX like UI design. You have:

1.Executable specifications

2.Performance measurements

3.Comparisons with official documentation

These are clear engineering outcomes that allow for objective comparison. For example, you can verify whether a program produces results within 5ms when executed.

Many epistemological arguments tend to overlook this.

A lot of code is open source. You can find mature implementations anywhere. Finding a repo from someone claiming 10 or 20 years of experience and running a differential comparison is not difficult at all."

nolist_policy an hour ago | parent | prev [-]

> As prompts get deeper, LLMs are already producing PhD-level code—and that's been shown in research.

This gave me a chuckle, "PhD-level code" is gross actually. Have you ever looked at the code of research papers?

jdw64 29 minutes ago | parent | next [-]

I've had opportunities to look at code from Korean and Chinese companies, not just academia. From what I've seen, AI generated code is often better than the average code in those codebases.

When I talk about PhD level code, I'm not referring to research code quality. I'm talking about research level algorithms.

Production code is built around reusability, policies, monadic chaining, and various code hygiene practices. Research code isn't. So what I'm saying is: 'Yes, I've collaborated with professors from top Korean universities.'

That said, I'm not writing PhD level code myself. The reason is that I take existing algorithms and apply code hygiene to them. I'm not actually developing new algorithms at a PhD level of understanding

So from what you're saying, it sounds like:

'Are you talking about those horrible research paper codes?'

But what exists in those research papers is: 'The domain modeling itself is difficult.'

I learned about code hygiene in production code—things like using functional approaches, and so on. Research code often lacks those things, so it might look bad. But that means the code quality is bad—not that the modeling itself is bad. And in fact, the core value is in being able to produce modeling at that level of difficulty.

In fact, most programmers, if trained like me, can do these things well enough. But domain modeling is different.

This touches on the distinction Frederick Brooks made in The Mythical Man-Month between 'Essential Complexity' and 'Accidental Complexity.' Code hygiene, functional chaining, monadic structures—these are techniques for reducing accidental complexity. With enough training, you can learn them. But domain modeling deals with essential complexity. It's about how difficult the problem itself is, and it's not easily solved through training alone. That's why the depth of modeling that researchers produce should be evaluated on a different axis from code quality.

This is what we call 'Essential Complexity.' From that perspective, saying code is 'good' usually means that the essential complexity is handled well on average. What I'm calling 'PhD-level' usually refers to modeling problems that are commonly difficult to solve.

jdw64 36 minutes ago | parent | prev [-]

I see it 'very often.' Most research paper code has no reusability—they just implement the algorithm.

So I can see why it might be considered low-quality. They don't use things like Result or policy types like we do in production code. But they're modifying or creating new algorithms, right?

The thing is, they don't need to do those things, so it looks lower-level—but the algorithm implementations themselves are actually pretty good, aren't they?

PhD thesis code often looks low-quality simply because there's no reason to reuse it like production code. The tensor computations inside are things that typical programmers can't easily do.

The point I'm making is that the core algorithmic work is something most programmers can't handle. And that's natural—highly skilled programmers have already built deep libraries for that.

When I talk about PhD-level code, I'm not referring to overall code quality. I'm talking about the level of the algorithms themselves.