> If an LLM is typing that code - and it can maintain a test suite that shows everything works correctly - maybe we don't need that abstraction after all.

I've had plenty of junior devs justify massive code bases of random scripts and 100+ line functions with the same logic. There's a reason senior devs almost always push back on this when it's encountered.

Everything hinges on that "if". But you're baking a tautology into your reasoning: "if LLMs can do everything we need them to, we can use LLMs for everything we need".

The reason we stop junior devs from going down this path is because experience teaches us that things will break and when they do, it will incur a world of pain.

So "LLM as abstraction" might be a possible future, but it assumes LLMs are significantly more capable than a junior dev at managing a growing mess of complex code.

This is clearly not the case with simplistic LLM usage today. "Ah! But you need agents and memory and context management, etc!" But all of these are abstractions. This is what I believe the parent comment is really pointing out.

If AI could do what we originally hoped it could: follow simple instructions to solve complex tasks. We'd be great, and I would agree with your argument. But we are very clearly not in that world. Especially since Karpathy can't even keep up with the sophisticated machinery necessary to properly orchestrate these tools. All of the people decrying "you're not doing it right!" are emphatically proving that LLMs cannot perform these tasks at the level we need them to.

> I'm now incentivized to use less abstractions.

I'm incentivised to use abstractions that are harder to learn, but execute faster or more safely once compiled. E.g. more Rust, Lean.

> If an LLM is typing that code - and it can maintain a test suite that shows everything works correctly - maybe we don't need that abstraction after all.

LLMs benefit from abstractions the same way as we do.

LLMs currently copy our approaches to solving problems and copy all the problems those approaches bring.

Letting LLMs skip all the abstractions is about as likely to succeed as genetic programming is efficient.

For example, writing more vanilla JS instead of React, you're just reinventing the necessary abstractions more verbosely and with a higher risk of duplicate code or mismatching abstractions.

In a recent interview with Bret Weinstein, a former professor of evolutionary biology, he proposed that one property of evolution that makes the story of one species evolving into another more likely is that it's not just random permutations of single genes; it's also permutations to counter variables encoded as telomeres and possibly microsatellites.

https://podcasts.happyscribe.com/the-joe-rogan-experience/24...

Bret compares this to flipping random bits in a program to make it work better vs. tweaking variables randomly in a high-level language. Mutating parameters at a high-level for something that already works is more likely to result in something else that works than mutating parameters at a low level.

So I believe LLMs benefit from high abstractions, like us.

We just need good ones; and good ones for us might not be the same as good ones for LLMs.

LLMs also have encyclopedic knowledge. Several times LLMs have found some huge block of code I wrote and reduced it down to a few lines. The other day they removed several thousand lines of brittle code I wrote previously for some API calls with a well-tested package I didn't know about. Literally thousands down to dozens.

My code is constantly shrinking, becoming better quality, more performant, more best-practice on a daily basis. And I'm learning like crazy. I'm constantly looking up changes it recommends to see why and what the reasons are behind them.

It can be a big damned dummy too, though. Just today it was proposing a massive server-side script to workaround an issue with my app I was deploying, when the actual solution was to just make a simple one-line change to the app. ("You're absolutely right!")

I find it interesting for your example you chose Moment.js -- a time library instead of something utilitarian like Lodash. For years I've following Jon Skeet's blog about implementing his time library NodaTime (a port of JodaTime). There are a crazy number of edge cases and many unintuitive things about modeling time within a computer.

If I just wanted the equivalent of Lodash's _.intersection() method, I get it. The requirements are pretty straightforward and I can verify the LLM code & tests myself. One less dependency is great. But with time, I know I don't know enough to verify the LLM's output.

Similar to encryption libraries, it's a common recommendation to leave time-based code to developers who live and breathe those black boxes. I trust the community verify the correctness of those concepts, something I can't do myself with LLM output.

For moment you an use `date-fns` and tree shake.

I'd rather have LLMs build on top of proven, battle-tested production libraries than keep writing their own from scratch. You're going to fill up context with all of its re-invented wheels when it already knows how to use common options.

Not to mention that testing things like this is hard. And why waste time (and context and complexity) for humans and LLMs trying to do something hard like state syncing when you can focus on something else?

Right there with you.

I'm instructing my agents to doing old school boring form POST, SSR templates, and vanilla JS / CSS.

I previously shifted away from this to abstractions because typing all the boilerplate was tedious.

But now that I'm not typing, the tedious but simple approach is great for the agent writing the code, and great for the the people doing code reviews.

> If an LLM is typing that code - and it can maintain a test suite that shows everything works correctly - maybe we don't need that abstraction after all.

But this is a highly non-trivial problem. How do you even possibly manually verify that the test suite is complete and tests all possible corner cases (of which there are so many because synchronizing state is a hard problem)?

At least React solves this problem in a non-stochastic, deterministic manner. What can be a good reason to replace something like React that works determinstically with LLM-assisted code that is generated stochastically and there's no easy way to manually verify if the implementation or the test suite is correct and complete?

> If an LLM is typing that code - and it can maintain a test suite that shows everything works correctly - maybe we don't need that abstraction after all.

for simple stuff, sure, React was ALWAYS inefficient. Even Javascript/client-side logic is still overkill a lot of the times except for that pesky "user expectations" thing.

for anything codebase that's long-lived and complex, combinatorics tells us how it'll near-impossible to have good+fast test coverage on all that.

part of the reason people don't roll their own is because being able to assume that the library won't have major bugs leads to an incredible reduction in necessary test service, and generally people have found it a safe-enough assumption.

throwing that out and trying to just cover the necessary stuff instead - because you're also throwing out your ability to quickly recognize risky changes since you aren't familiar with all the code - has a high chance of painting you into messy corners.

"just hire a thousand low-skilled people and force them to write tests" had more problems as a hiring plan then just "people are expensive."

I don't trust LLM enough to handle the maintenance of all the abstraction buried in react / similar library. I caught some of the LLMs taking nasty shortcuts (e.g. removing test constraints or validations in order to make the test green). Multiple times. Which completely breaks trust.

And if I have to closely supervise every single change, I don't believe my development process will be any better. If not worse.

Let alone new engineers who join the team and all of a sudden have to deal with a unique solution layer which doesn't exist anywhere else.

> Why do we code with React?

...is a loaded question, with a complex and nuanced answer. Especially when you continue:

> it's worth paying the React complexity/page-weight tax

All right; then why do we code in React when a smaller alternative, such as Preact, exists, which solves the same problem, but for a much lower page-weight tax?

Why do we code in React when a mechanism to synchronize data with tiny UI fragments through signals exists, as exemplified by Solid?

Why do people use React to code things where data doesn't even change, or changes so little that to sync it with the UI does not present any challenge whatsoever, such as blogs or landing pages?

I don't think the question 'why do we code with React?' has a simple and satisfactory answer anymore. I am sure marketing and educational practices play a large role in it.

Has anyone tried the experiment that is sort of implied here? I was wondering earlier today, what it would be like to pick a simple app, pick on OS, and just tell an LLM to write that app using only machine code and native ADKs, and skip all intermediate layers?

We seem to have created a large bureaucracy for software development, where telling a computer how to execute an app involves keeping a lot of cogs in a big complicated machine happy. But why use the automation to just roll the cogs? Why not just simplify/streamline? Does an LLM need to worry about using the latest and greatest abstractions? I have to assume this has been tried already...

If you work at a megacorp right now, you know whats happening isn't people deciding to use less libraries. It's developers being measured by their lines of code, and the more AI you use the more lines of code and 'features' you can ship.

However, the quality of this code is fucking terrible, no one is reading what they push deeply, and these models don't have enough 'sense' to make really robust and effective test suites. Even if they did, a comprehensive test suite is not the solution to poorly designed code, it's a band aid -- and an expensive one at scale.

Most likely we will see some disasters happening in the next few years due to this mode of software development, and only then will people understand to use these agents as tools and not replacements.

...Or maybe we'll get AGI and it will fix/maintain the trash going out there today.

If LLMs are that capable, then why are AI companies selling access to them instead of using them to conquer markets?

Huh, I've been assuming the opposite: better to use React even if you don't need it, because of its prevalence in the training data. Is it not the case that LLMs are better at standard stacks like that than custom JS?

Why would I want to maintain in perpetuity random snippets when a library exists? How is that an improvement?

I'd rather use React than a bespoke solution created by an ephemeral agent, and I'd rather self-trepanate than use React

  > I'm now incentivized to use less abstractions.

Nutty idea: train on ASM code. Create an LLM that compiles prompts directly to machine code.

The problem is, what do you do _when_ it fails? Not "if", but "when".

Can you manually wade through thousands of functions and fix the issue?

> and it can maintain a test suite that shows everything works correctly

Are you able to efficiently verify that the test suite is testing what it should be testing? (I would not count "manually reviewing all the test code" as efficient if you have a similar amount of test code to actual code.)

Sometimes a change to the code under test means that a (perhaps unavoidably brittle) test needs to be changed. In this case, the LLM should change the test to match the behaviour of the code under test. Other times, a change to the code under test represents a bug that a failing test should catch -- in this case, the LLM should fix the code under test, and leave the test unchanged. How do you have confidence that the LLM chooses the right path in each case?

That's a fundamental misunderstanding

The role of abstractions *IS* to prevent (eg "compress") the need for a test suite, because you have an easy model to understand and reason about

disruption is a code word for deregulation, and deregulation is bad for everyone except execs and investors

One possible positive outcome of all this could be sending LLMs to clean up oceans of low value tech debt. Let the humans move fast, let the machines straighten out and tidy up.

The ROI of doing this is weak because of how long it takes an expensive human. But if you could clean it up more cheaply, the ROI strengthens considerably- and there’s a lot of it.

It's not something that suddenly changed. "I'll generate some code" is as nondeterministic as "I'll look for a library that does it", "I'll assign John to code this feature", or "I'll outsource this code to a consulting company". Even if you write yourself, you're pretty nondeterministic in your results - you're not going to write exactly the same code to solve a problem, even if you explicitly try.

Why would the average programmer have a problem with it?

The average programmer is already being pushed into doing a lot of things they're unhappy about in their day jobs.

Crappy designs, stupid products, tracking, privacy violation, security issues, slowness on customer machines, terrible tooling, crappy dependencies, horrible culture, pointless nitpicks in code reviews.

Half of HN is gonna defend one thing above or the other because $$$.

What's one more thing?

It's wild that management would be willing to accept it.

I think that for some people it is harder to reason about determinism because it is similar to correctness, and correctness can, in many scenarios be something you trade off - for example in relation to scaling and speed you will often trade off correctness.

If you do not think clearly about the difference with determinism and other similar properties like (real-time) correctness which you might be willing to trade off, you might think that trading off determinism is just more of the same.

Note: I'm against trading off determinism, but I am willing to think there might be a reason to trade it off, just I worry that people are not actually thinking through what it is they're trading when they do it.

I think those that are most successful at creating maintainable code with AI are those that spend more time upfront limiting the nondeterminism aspect using design and context.

Mortgages don't pay for themselves.

It's not that wild. I like building things. I like programming too, but less than building things.

> It’s wild that programmers are willing to accept less determinism.

It's wild that you think programmers is some kind of caste that makes any decisions.

You can have the best of both worlds if you use structured/constrained generation.

The good ones don't accept. Sadly there's just many more idiots out there trying to make a quick buck

There has always been a laissez-faire subset of programmers who thrive on living in the debugger, getting occasional dopamine hits every time they remove any footgun they previously placed.

I cannot count the times that I've had essentially this conversation:

"If x happens, then y, and z, it will crash here."

"What are the odds of that happening?"

"If you can even ask that question, the probability that it will occur at a customer site somewhere sometime approaches one."

It's completely crazy. I've had variants on the conversation from hardware designers, too. One time, I was asked to torture a UART, since we had shipped a broken one. (I normally build stuff, but I am your go-to whitebox tester, because I hone in on things that look suspicious rather than shying away from them.) When I was asked the inevitable "Could that really happen in a customer system?" after creating a synthetic scenario where the UART and DMA together failed, my response was:

"I don't know. You have two choices. Either fix it where the test passes, or prove that no customer could ever inadvertently recreate the test conditions."

He fixed it, but not without a lot of grumbling.

I mean we've had to cope with users for ages, this is not that different.

This gets repeated all the time, but it’s total nonsense. The output of an LLM is fixed just as the output of a human is.

I have to admit, LLMs do save a lot of typing a d associated syntax errors. If you know what you want and can spot and fix mistakes made by the LLM then they can be pretty useful. I don’t think it’s wise to use them for development if you are not knowledgeable enough in the domain and language to recognize errors or dead ends in the generated code though.

I'm also interested in hearing this.

For me, I'm planning to ride out this industry for another couple years building cash until I can't stand it, then pivot to driving a city bus.

The system is designed to do exactly that. This is called ‘productivity increase’ and is deflationary in large dosages. Deflation sounds good until you understand where it’s coming from.

People keep making this argument, but the jump to LLM driven development is such a conceptually different thing than any previous abstraction

This is true, though the people that actually push the field forward do know enough about every level of abstraction to get the job done. Making something (very important) horrible just to rush to market can be a pretty big progress blocker.

Jensen is someone I trust to understand the business side and some of those lower technical layers, so I'm not too concerned.

And if you're writing machine code directly, you're still relying on about ten layers of abstraction that the wizards at the chip design firms have built for you.