All I want is an explanation of how you're all reliably getting good enough (or better) results from LLMs. Surely us AI critics are doing something wrong, something much more specific than needing to "prompt harder", if we accept the framing that 90% of world class programmings are achieving that, while the rest of us are not. There simply has to be some specific, repeatable, and yes, teachable thing we're missing. So what is it?

▲

sadilek 4 hours ago | parent | next [-]

Try requesting a feature from Opus 5.5 high that would take you a day or two to implement and test well. It'll work for 20-30 minutes or so. Then ask it to review the changes with a sub agent of itself and with Codex Sol 6.1 high. Then let it address the findings and iterate the review/fix cycle until the reviewers are happy.

▲

ObiKenobi 2 hours ago | parent | prev | next [-]

There are Addons for IDEs like "AI-Autocomplete". When you don't want the AI to write whole features at once, you can prompt it to write the functions one by one and work like that. Still a lot faster then writing all functions by yourself and granular enough that it is easy to describe and easy to review.

▲

baggy_trough 2 hours ago | parent | prev | next [-]

My best explanation is that AI coding critics have a kind of willful blindness or fall prey to motivated reasoning.

▲

unethical_ban 2 hours ago | parent | prev | next [-]

Anecdotally, for small tools and projects, it can crank out functional tools that would take me days or weeks to remember how to do manually. Every time I ask LLM to write something, it also writes tests and validates its code against known values. I additionally spot-check the LLM's code against my own known values in prod, and have had to correct it on some functions. The code I write is mainly analysis of logs and configurations, and they seem well trained on the products that I am using.

Testing. It's all about having unit tests, and known outcomes that the LLM can check against every time it makes changes. For my work, that is good enough. For safety-critical systems I would demand more rigor in the testing and the human review and creation of tests.

▲

koonsolo 5 hours ago | parent | prev [-]

Most senior developers that steered juniors or interns in the past, are able to work with LLM's.

Sure, LLM's are unreliable. So are juniors.

Sure, LLM's code quality is not always great. Well, so is the juniors code.

The benefit of LLM's is that they can work super fast and have a vast amount of knowledge. The benefit of juniors is that they actually learn from their mistakes and get better over time. LLM's stay at intern/junior level, even with the correct context.

So for me, I see each LLM session as a new intern coming in. You hand them the right context, give them the correct instruction, and off they go. Then they come back, and you will have to judge how detailed you want to review what they produced. Or in other words, can you get away with some bad code? And for the next session, it's a new intern that starts from 0.

But it is also highly dependent on the project you are working on. If it's a big complex codebase, LLM's will struggle, and they can only do simple tasks (however, they can help with testing a lot!)

So the teachable thing is unfortunately experience:

- Know where quality matters, and where it doesn't

- Know what kind of mistakes could be made and should be checked for

- Know which things can be duck-taped together, because they live at the edge, and which things need a proper foundation, because the rest builts on it.

So in the end, I think it all comes down to making the right trade-offs and the right decisions (which LLM's are terrible at, by the way). And unfortunately that is gained by the slow process of experience.

I think it's very tough for junior developers right now, because they either go fast by letting the LLM's do the work, but then they learn nothing. Or they go slow by asking the LLM's what's going on, and then do learn.

A senior developer never understood the full code-base, and never reviewed every line of code. That's just impossible. And now every developer needs to be a senior developer that is able to steer these new "interns".

And just like with juniors and interns, you have to put a lot of effort into it, and it's not always clear in the end if you gained or lost time :D.

▲

popkkksidhd an hour ago | parent | next [-]

> A senior developer never understood the full code-base, and never reviewed every line of code.

No, but when a junior puts up a PR I do. They aren’t trustworthy enough yet.

One of the biggest parts of being senior is ownership of your code and knowing you’re putting your trustworthiness behind every PR you make. If you’re making bad PRs, you can’t say it’s Claude’s fault because you didn’t think a certain area was worth reviewing.

> That's just impossible.

Somewhat, but this is directionally bad advice. There’s huge payoffs for taking time to thoroughly understand areas you’re working in. It greatly amplifies your ability to prompt and overall execute in a codebase.

▲

em-bee 4 hours ago | parent | prev [-]

LLMs and juniors are not the same. i made this argument on a previous discussion of the same topic: https://news.ycombinator.com/item?id=48212898

▲

unethical_ban 2 hours ago | parent | next [-]

One thing that stood out in that comment: You said "you like" the act of programming elegantly and understanding the problem.

There's nothing wrong with that. Many of us do (or did) like programming and the problem-solving itch it scratched for us. But getting work done for pay is more than what we like. If an LLM can reliably solve a problem for me in a day that would have taken a project, a team of several people, and close to a month of work to build out manually, then I go with the tool that works.

And that "if" is the reality I live in. About a decade ago, I wrote a program to do analysis of config files for a product our team uses. It took me weeks of side-work to implement the idea of passing boolean python functions as checks for the configuration. Such an elegant way of creating new checks on our system! It was company work, and when I left to a new place, I didn't have that code, just the idea. Fast forward five years and I want to write the same thing at a new place. I asked an LLM to do the same task, and it wrote exactly my solution immediately. It just worked. It shocked me. In the last few months, I've had it write analysis tools against this product that would have taken me days or weeks. I am admittedly not a comp-sci level programmer.

There is no denying that LLMs are powerful coding tools. That can be true at the same time that we lament the end of manual-coding-for-pay, the death of programming education as we know it, and the worries about employment and society.

▲

koonsolo 4 hours ago | parent | prev [-]

I don't claim LLM's and juniors are the same. I claim that working with LLM's and juniors are similar. If you are able to work with juniors, you are probably able to work with LLM's.

To comment on the post you linked:

I agree that people can be thought. But on the other hand, I also had juniors that reached their limit pretty fast, which can also be frustrating. The best way to "teach" LLM's is the context, but I agree it has limits and is not really teaching anything, just steering them in the right direction every time (like a checklist).

Your comment basically states that running LLM's is a gamble, but according to my experience, most of the time it delivers what I ask. Maybe in a different way that how I would do it, but I want results, and am not interested in micro-managing.

Are you working on a project that requires high quality code? Or is it your preference to work in such a codebase? For me, I was always ok with "good enough" code. And sometimes "good enough" needs to be good, and other times you can get away with some really bad code.

	▲	em-bee 36 minutes ago \| parent [-]
		I don't claim LLM's and juniors are the same. I claim that working with LLM's and juniors are similar. i meant "working with" as i explained in the linked comment. the feedback and response i get from juniors is very different. and yes, sure every junior dev is different from the others, but i simply enjoy working with people and i can't stand working with machines in the same way. what bothers me about LLMs is that they are not deterministic. that just drives me nuts. i do enjoy learning the ins and outs of a language or compiler to figure out how to write the code to get the results i need. but LLMs just don't work like that. the same instruction that produced good working code today could produce nonsense tomorrow. this is not about quality. this is about trust. i can't trust LLMs and that makes it very uncomfortable for me to work with them.