Remix.run Logo
baxuz 2 days ago

The thing is that the data from actual research doesn't support your anecdotal proof of quality:

- https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...

- https://www.theregister.com/2025/06/29/ai_agents_fail_a_lot/

But more importantly, it makes you stupid:

- https://www.404media.co/microsoft-study-finds-ai-makes-human...

- https://archive.is/M3lCG

And it's an unsustainable bubble and wishful thinking, much like crypto:

- https://dmitriid.com/everything-around-llms-is-still-magical...

So while it may be a fun toy for senior devs that know what to look for, it actually makes them slower and stupider, making them progressively less capable to do their job and apply critical thinking skills.

And as for juniors — they should steer clear from AI tools as they can't assess the quality of the output, they learn nothing, and they also get critical thinking skills impaired.

So with that in mind — Who is the product (LLM coding tools) actually for, and what is its purpose?

I'm not even going into the moral, ethical, legal, social and ecological implications of offloading your critical thinking skills to a mega-corporation, which can only end up like https://youtu.be/LXzJR7K0wK0

com2kid 2 days ago | parent | next [-]

All of those studies have been torn apart in detail, often right here on HN.

> So while it may be a fun toy for senior devs that know what to look for, it actually makes them slower and stupider, making them progressively less capable to do their job and apply critical thinking skills.

I've been able to tackle problems that I literally would not have been able to undertake w/o LLMs. LLMs are great at wading through SO posts and GH issue threads and figuring out what magic set of incantations makes some stupid library actually function. They are really good at writing mock classes way faster than I ever have been able to. There is a cost/benefit analysis for undertaking new projects, and if "minor win" involves days of wading through garbage, odds are the work isn't going to happen. But with LLMs I can outsource the drudgery part of the job (throwing crap tons of different parameters at a poorly documented function and seeing what happens), and actually do the part that is valuable (designing software).

You still have to guide the design! Anyone letting LLMs design software is going to fail hard, LLMs still write some wacky stuff. And they are going to destroy juniors, I don't know what the future of the field is going to be like (not pretty that is for sure...)

But I just had an LLM write me a script in ~2 minutes (me describing the problem) that would've taken me 30-60 minutes to write and debug. There would have been no "learning" going on writing a DOS batch script (something I have to do once very 2 or 3 years, so I forget everything I know each time).

blub 2 days ago | parent | next [-]

The AI in OSS study was not “torn apart”.

The AI aficionados made scary faces at it, tried to scratch it with their cute little claws and then gave up and stopped talking about it. :)

otabdeveloper4 a day ago | parent [-]

Maybe try asking ChatGPT to debunk this study?

Tainnor 2 days ago | parent | prev [-]

> All of those studies have been torn apart in detail, often right here on HN.

You mean the same Hacker News where everyone was suddenly an expert in epidemiology a few years ago and now can speak with authority to geopolitics?

com2kid 2 days ago | parent [-]

Except we are experts on programming, and on the development and deployment of new technologies.

"Large group of experts software engineers have informes opinions on software engineering" isn't exactly a controversial headline.

WesolyKubeczek 2 days ago | parent | next [-]

Given what the parent comment is saying, I'm now doubting if "expertise in programming" is not just LARPing too. A handful of people actually know how to do it, and the rest of commenters engage in self-aggrandizement.

Tainnor 16 hours ago | parent | prev [-]

> Except we are experts on programming

Which is not the same thing as being able to judge a study design.

wilson090 2 days ago | parent | prev | next [-]

These studies profoundly miss the mark and were clearly written for engagement/to push a certain view. It's abundantly clear to any developer who has used LLMs that they are a useful tool and have turned the corner in terms of the value they're able to provide vs their limitations.

cess11 2 days ago | parent [-]

Not to me. I have also not seen any signs that this technology has had macroeconomic effects, and I don't know of any developers in meatspace that are impressed.

To me it seems like a bunch of religious freaks and psychopaths rolled out a weird cult, in part to plaster over layoffs for tax reasons.

oytis 2 days ago | parent | next [-]

> I don't know of any developers in meatspace that are impressed

I have a theory that there is some anomaly around Bay Area that makes LLMs much better there. Unfortunately the effects seem to be not observable from the outside, it doesn't seem to work on anything open source

bcrosby95 2 days ago | parent | prev | next [-]

My boss was puzzled that despite LLMs writing ~30% of our code, he's not seeing a 30% increase in efficiency. Strange, that is.

johnb231 2 days ago | parent [-]

Devs finish the work 30% faster and take the rest of the day off. That's what I would do. Working remotely.

cess11 2 days ago | parent [-]

People aren't generally able to keep up the discipline to time when to pass on tickets to hide changes in their ability, unless it's forced by a constant anxiety.

Developers are also not very good at estimating how long something is supposed to take. If there was even a 10% jump in profitability in the software department it would have been obvious to bean counters and managers. You'd also see a massive recruitment spree, because large organisations ramp up activities that make money in the short term.

wilson090 2 days ago | parent | prev [-]

The anti-LLM crowd on HN is far more cultish. I don't know why some developers insist on putting their head in the sand on this.

Jensson 2 days ago | parent | next [-]

If LLM makes your coworkers slower why should you worry?

zer00eyz 2 days ago | parent | prev | next [-]

The pro-LLM crowd on HN is just as cultish. The divide is as diverse as the work we do:

There is work that I do that is creative, dynamic and "new". The LLM isn't very helpful at doing that work. In fact it's pretty bad at getting that sort of thing "right" at all. There is also plenty of work that I do that is just transformational, or boiler plate or a gluing this to that. Here the LLM shines and makes my job easy by doing lots of the boring work.

Personal and professional context are going to drive that LLM experience. That context matters more than the model ever will. I would bet that there is a strong correlation between what you do day to day and how you feel about the quality of LLM's output.

skydhash a day ago | parent [-]

What is the thing about glue code that people are rambling about? I’ve never seen such glue code that is tedious to write. What I’ve seen are code examples that I copy-pasted, code generators that I’ve used, and snippets that I’ve inserted. I strongly suspect that the tediousness was about making these work (aka understanding), not actually typing the code.

zer00eyz a day ago | parent [-]

> I’ve never seen such glue code that is tedious to write.

Its a fair point, its not the writing per se thats tedious:

Fetch data from API 9522, write storage/trasformation/validation code, write display code. Test, tweak/fix, deploy.

Do you know how many badly designed and poorly documented API's I have had to go through in 25+ years? Do you know how many times I have written the same name/first_name/FirstName/First_name mapping between what comes in and what already exists. Today it's an old personal project, tommrow a client app, the day after home assistant (and templated yaml).

Why should I spend any time figuring out if the api doc is poorly or well written? Why should I learn what esoteric scheme of tokens you have chosen to put up the facade of security. Is mapping code fun to write? It's like the boiler plate around handling an error or writing a log message (things that you let autocomplete do if you can). Do you really want to invest in the bizarre choices of systems you USE but not often enough to make it worth your time to commit their goofy choices to memory (I'm looking at you templated yaml).

You are right that the "code is easy". It's the whole process and expense of brain power on things that are, in the long run, useless that makes it tedious. The study where people did not retain what the wrote/did with the LLM is a selling point not a down side. Tomorrow I have to do the same with API 9523 and 9524, and I'm going to be happy if it gets done and I retain none of it.

cess11 a day ago | parent [-]

I quite enjoy inventing parsers for docs and generating clients. You should try that approach instead of writing everything by hand.

cess11 2 days ago | parent | prev | next [-]

On what, exactly? Where are the measurable gains?

I've tried out a lot of angles on LLM:s and besides first pass translations and audio transcriptions I have a hard time finding any use for them that is a good fit for me. In coding I've already generated scaffolding and CRUD stuff, and typically write my code in a way that makes certain errors impossible where I actually put my engineering while the assistant insists on adding checks for those errors anyway.

That's why I gave up on Aider and pushing contexts into LLM:s in Zed. As far as I can tell this is an unsolvable problem currently, the assistant would need to have a separate logic engine on the AST and basically work as a slow type checker.

Fancy autocomplete commonly insists on using variables that are previously unused or make overly complicated suggestions. This goes for both local models and whatever Jetbrains pushed out in IDEA Ultimate. One could argue that I'm doing it wrong but I like declaring my data first and then write the logic which means there might be three to ten data points lingering unused in the beginning of a function while I'm writing my initial implementation. I've tried to wriggle around this by writing explicit comments and so on but it doesn't seem to work. To me it's also often important to have simple, rather verbose code that is trivial to step or log into, and fancy autocomplete typically just don't do this.

I've also found that it takes more words to force models into outputting the kind of code I want, e.g. slurp the entire file that is absolutely sure to exist and if it doesn't we need to nuke anyway, instead of five step read configured old school C-like file handles. This problem seems worse in PHP than Python, but I don't like Python and if I use it I'll be doing it inside Elixir anyway so I need to manually make sure quotations don't break the Elixir string.

Personally I also don't have the time to wait for LLM:s. I'm in a hurry when I write my code, it's like I'm jogging through it, because I've likely done the thinking and planning ahead of writing, so I just want to push out the code and execute it often in a tight cycle. Shutting down for twenty to three hundred seconds while the silly oracle is drawing power over and over again is really annoying. Like, I commonly put a watch -n on the test runner in a side terminal with usually 3-10 seconds depending on how slow it feels at the moment, and that's a cadence LLM:s don't seem to be able to keep up with.

Maybe the SaaS ones are faster but for one I don't use them for legal reasons and secondly every video of one that I watch is either excruciatingly slow or they snipped or sped up 'thinking' portions. Some people seem to substitute for people and chat with their LLM:s like I would with a coworker or expert in some subject, which I'm not interested in, in part because I fiercely dislike the 'personality' LLM:s usually emulate. They are also not knowledgeable in my main problem domains and can't learn, unlike a person, whom I could explain context and constraints to before we get to the part where I'm unsure or not good enough.

To me these products are reminiscent of Wordpress. They might enable people like https://xcancel.com/leojr94_ to create plugins or prototypes, and some people seem to be able to maintain small, non-commercial software tools with them, but it doesn't seem like they're very good leverage for people that work on big software. Enterprise, critical, original systems, that kind of thing.

Edit: Related to that, I sometimes do a one-shot HTML file generation because I suck at stuff like Tailwind and post-HTML4 practices, and then I paste in the actual information and move things around. Seems fine for that, but I could just script it and then I'd learn more.

leptons 2 days ago | parent | prev [-]

> I don't know why some developers insist on putting their head in the sand on this.

You don't think we're not using "AI" too? We're using these tools, but we can see pretty clearly how they aren't really the boon they are being hyped-up to be.

The LLM is kind of like a dog. I was trying to get my dog to do a sequence of things - pick up the toy we were playing with and bring it over to me. He did it a couple of times, but then after trying to explain what I wanted yet again, he went and picked up a different toy and brought it over. That's almost what I wanted.

Then I realized that matches the experience I've had with various "AI" coding tools.

I have to spend so much time reading and correcting the "AI" generated code, when I could have just coded the same thing myself correctly the first time. And this never stops with the "AI". At least with my dog, he is very food motivated and he learns the tricks like his life depends on it. The LLM, not so much.

sharkjacobs 2 days ago | parent | prev | next [-]

- higher editorial standards and gatekeeping meant print media was generally of higher quality than internet publications

- print publications built reputations of spans of time that the internet still hasn't existed for, earning greater trust and authority, and helping to establish shared cultural touchstones and social cohesion

- copyright was clearer and more meaningful, piracy was more difficult

- selling physical copies and subscriptions was a more stable revenue source for creators and publishers than the tumult of selling ads in the 21st century

And all of this was nothing in the face of "receiving pages of text. Faster than one could read"

estsauver 2 days ago | parent | prev | next [-]

I think it's worth saying that I basically completely disagree with your assessment (how you read the evidence, your conclusions, and quite possibly your worldview,) and think that if you were to give me access to infinite throughput claude code in 2018 that I could have literally ruled the world.

I'm not the most impressive person on hacker news by a wide margin, but I've built some cool things that were hard, and I think they are absolutely inevitable and frequently have the exact same "one shot" type experience where things just work. I would seriously reconsider whether it is something that you can't make work well for you, or something you don't want to work well.

mns 2 days ago | parent | prev | next [-]

> So with that in mind — Who is the product (LLM coding tools) actually for, and what is its purpose?

It's for grifters to make more money by getting viral on Twitter and non technical managers that want to get rid of their workforce.

12345hn6789 a day ago | parent | prev | next [-]

N=16

theodric 2 days ago | parent | prev | next [-]

> Who is the product (LLM coding tools) actually for, and what is its purpose?

Ideally: it's for people who aren't devs, don't want to be devs, can't afford to pay devs to build their hobby projects for them, and just want to have small tools to unblock or do cool stuff. It's pretty incredible what a no-coder can knock off in an evening just by yelling at Cursor. It's a 3D printer for code.

But realistically, we know that the actual answer is: the people who already destroy companies for their own short-term benefit and regard all tech workers as fungible resources will have no problem undermining the feasibility of hiring good senior devs in 2050 in exchange for saving a ton of money now by paying non-devs non-dev money to replace juniors, leaning HARD on the remaining meds/seniors to clean up the resulting mess, and then pulling the ripcord on their golden parachute and fucking off to some yacht or island or their next C-suite grift before the negative consequences hit, all the while touting all the money they saved "automating" the development process at their last corp. And then private equity buys it up, "makes it efficient" to death, and feeds its remaining viable organs to another company in their portfolio.

johnb231 2 days ago | parent | prev | next [-]

Dude in 1972 looking at Atari Pong: “computer graphics will never achieve realism”

handoflixue 2 days ago | parent | prev [-]

"But more importantly, it makes you stupid:"

I don't think it was your intent, but that reads out as a seriously uncalled for attack - you might want to work on your phrasing. Hacker News rules are pretty clear on civility being an important virtue.

baxuz a day ago | parent | next [-]

I didn't target the author, and I used the terminology used in the article heading

tempfile 2 days ago | parent | prev [-]

I doubt it. It's not directed at an individual, and it's presented as a passive fact. It's a bit like saying "drugs make you stupid", which no-one would complain about.