> On the other hand, if you are about to be unemployed, this is less impressive

> salesman can convince executives that it does

I tend to think that reality will temper this trend as the results develop. Replacing 10 engineers with one engineer using Cursor will result in a vast velocity hit. Replacing 5 engineers with 5 "agents" assigned to autonomously implement features will result in a mess eventually. (With current technology -- I have no idea what even 2027 AI will do). At that point those unemployed engineers will find their phones ringing off the hook to come and clean up the mess.

Not that unlike what happens in many situations where they fire teams and offshore the whole thing to a team of average developers 180 degrees of longitude away who don't have any domain knowledge of the business or connections to the stakeholders. The pendulum swings back in the other direction.

I just think Jevins paradox [1]/Gustafson's Law [2] kind of applies here.

Maybe I shouldn't have used the word "replaced", as I don't really think it's actually going to "replace" people long term. I think it's likely to just lead to higher output as these get better and better .

[1] https://en.wikipedia.org/wiki/Jevons_paradox

[2] https://en.wikipedia.org/wiki/Gustafson%27s_law

In my experience it has gotten considerably better. When I get it to generate C, it often gets the pointer logic correct, which wasn't the case three years ago. Three years ago, ChatGPT would struggle with even fairly straightforward LaTeX, but now I can pretty easily get it to generate pretty elaborate LaTeX and I have even had good success generating LuaTeX. I've been able to fairly successfully have it generate TLA+ spec from existing code now, which didn't work even a year ago when I tried it.

Of course, sample size of one, so if you haven't gotten those results then fair enough, but I've at least observed it getting a lot better.

It seems like just such a weird and rigid way to evaluate it? I am a somewhat reasonable human coder, but I can't copy and paste a bunch of code without alterations from memory either. Can someone still find a use for me?

I think what they're best at right now is the initial scaffolding work of projects. A lot of the annoying bootstrap shit that I hate doing is actually generally handled really well by Codex.

I agree that there's definitely some overhype to them right now. At least for the stuff I've done they have gotten considerably better though, to a point where the code it generates is often usable, if sub-optimal.

For example, about three years ago, I was trying to get ChatGPT to write me a C program to do a fairly basic ZeroMQ program. It generated something that looked correct, but it would crash pretty much immediately, because it kept trying to use a pointer after free.

I tried the same thing again with Codex about a week ago, and it worked out of the box, and I was even able to get it to do more stuff.

For a long time, I've wanted to write a blog post on why programmers don't understand the utility of LLMs[1], whereas non-programmers easily see it. But I struggle to articulate it well.

The gist is this: Programmers view computers as deterministic. They can't tolerate a tool that behaves differently from run to run. They have a very binary view of the world: If it can't satisfy this "basic" requirement, it's crap.

Programmers have made their career (and possibly life) being experts at solving problems that greatly benefit from determinism. A problem that doesn't - well either that needs to be solved by sophisticated machine learning, or by a human. They're trained on essentially ignoring those problems - it's not their expertise.

And so they get really thrown off when people use computers in a nondeterministic way to solve a deterministic problem.

For everyone else, the world, and its solutions, are mostly non-deterministic. When they solve a problem, or when they pay people to solve a problem, the guarantees are much lower. They don't expect perfection every time.

When a normal human asks a programmer to make a change, they understand that communication is lossy, and even if it isn't, programmers make mistakes.

Using a tool like an LLM is like any other tool. Or like asking any other human to do something.

For programmers, it's a cardinal sin if the tool is unpredictable. So they dismiss it. For everyone else, it's just another tool. They embrace it.

[1] This, of course, is changing as they become better at coding.

> What, precisely, are they good for?

scamming people

Its strong enough to replace humans at their jobs and weak enough that it cant do basic things. Its a paradox. Just learn to be productive with them. Pay $200/month and work around with its little quirks. /s

There's no silver bullet. I’m not a researcher, but I’ve done my best to understand how these systems work—through books, video courses, and even taking underpaid hourly work at a company that creates datasets for RLHF. I spent my days fixing bugs step-by-step, writing notes like, “Hmm… this version of the library doesn’t support protocol Y version 4423123423. We need to update it, then refactor the code so we instantiate ‘blah’ and pass it to ‘foo’ before we can connect.”

That experience gave me a deep appreciation for how incredible LLMs are and the amazing software they can power—but it also completely demystified them. So by all means, let’s use them. But let’s also understand there are no miracles here. Go back to Shannon’s papers from the ’60s, and you'll understand that what seems to you like "emerging behaviors" are quite explainable from an information theory background. Learn how these models are built. Keep up with the latests research papers. If you do, you’ll recognize their limitations before those limitations catch you by surprise.

There is no silver bullet. And if you think you’ve found one, you’re in for a world of pain. Worse still, you’ll never realize the full potential of these tools, because you won’t understand their constraints, their limits, or their pitfalls.

Maybe CEOs should face consequences for going on the stage and outwardly lying. Instead they're rewarded by a bump in stock price because people appear to have amnesia.