So you're saying that it depends whether you're measuring "success at task X" (where in average everyone improves) vs comparative success (where people with knowledge can do far and away better at specific tasks).

Corollary to this is an important part of LLM usage is what I call pinning it to reality. That is, designing verification steps that interact with the real world in some way not easy to hallucinate or work around. This means things like having code that interacts with the physical world, round trip tests, arriving at the same result using different paths, interoperability / replication with external libraries / competing products, performance improvement projects that start with robust performance test suites, and similar sorts of things that reduce to "how do I provide evidence that's difficult to fool myself about".

This includes things like "before you start fixing this bug, write two tests that fail proving it exists".

Expertise is good, but a wise expert will set up methods for the machine to prove to itself that a desired result is achieved removing the expert from the tight development loop.

Yeah, the people who say no expertise is needed for these things confuse me somewhat. This is indeed the case if you want to be a meat wrapper around an LLM, understanding neither your inputs nor your outputs. But at that point, what is the point of you versus going to the LLM myself? Expertise is necessary because it adds understanding and structure to the blob of text produced by an LLM. Progress can only be built on such understanding.

I am tempted to say (uncharitably) that the 'No knowledge needed! Just add LLMs!' byline is wishful thinking by non-experts who do not want to confront the reality that they will ultimately need to learn things.

It's actually refreshing when a model is sure about something because it actually tested it and has the receipts. Opus 5 seems really good about testing its own knowledge with experiments. Scientific method ftw.

There was one math proof that was AI generated going around Twitter and the chat transcript to generate it was basically the human prompting "keep going" until it solved it.

Tao's chat was fascinating because the questions he was asking belied expert knowledge of the subject that only a handful of people could have asked.

[dead]

He's on to us, boys! Call in a code 43.

(Sorry, I'm in a crappy mood, but what on Earth are we supposed to take away from this? Everyone who disagrees with you is secretly an idiot, or worse, they're smart enough to know they're idiots but too proud to admit it?)

On a more helpful note, I think your "confusion" if honest can probably be resolved by realizing that "skeptics" are not a monolith.

I think it's the same thing as it was with googling. "Oh you're such a wizard problem solver, how did you fix my printer??" "I just typed the most important words about the problem as you told it to me, the printer's model number and 'solution' and a forum link was on the first page, searched the page for 'solved' and followed the 3 simple steps of which buttons to press."

Then you saw how other people used Google, by filling the search bar with utterly irrelevant words, missing the one key word that's most important to what they are trying to do, then not be able to evaluate the returned search results and triage for which is most "solution-shaped", and they get drawn into wrong search hits, reading a clearly irrelevant page instead of quickly backing out to the search results page to try another page etc.

Or see how people couldn't formulate questions on StackOverflow, other than dumping a huge code chunk and saying "it doesn't work".

Now, AI makes these easier. You can now really just type natural language into the textbox, not just key words, you don't have to know about quote marks and plus signs etc. You can paste the code and say it doesn't work, and the AI just might actually spot a bug.

But having general problem solving common sense will still give you very good dividends.

This is becoming such a common statement on here it’s almost copy pasta.

Maybe the answer is more along the lines of “people are using them for different things and getting different results”?

Why does it have to be snark and “these people must be stupid”