I like this article, and I didn't expect to because there's been volumes written about how you should be boring and building things in an interesting way just for the hell of it, is bad (something I don't agree with).

Small models doing interesting (boring to the author) use-cases is a fine frontier!

I don't agree at all with this though:

> "LLMs are not intelligent and they never will be."

LLMs already write code better than most humans. The problem is we expect them to one-shot things that a human may spend many hours/days/weeks/months doing. We're lacking coordination for long-term LLM work. The models themselves are probably even more powerful than we realize, we just need to get them to "think" as long as a human would.

> LLMs already write code better than most humans.

If you mean better than most humans considering the set of all humans, sure. But they write code worse than most humans who have learned how to write code. That's not very promising for them developing intelligence.

The issue is one that's been stated here before: LLMs are language models. They are not world models. They are not problem models. They do not actually understand world or the underlying entities represented by language, or the problems being addressed. LLMs understand the shape of a correct answer, and how the components of language fit together to form a correct answer. They do that because they have seen enough language to know what correct answers look like.

In human terms, we would call that knowing how to bullshit. But just like a college student hitting junior year, sooner or later you'll learn that bullshitting only gets you so far.

That's what we've really done. We've taught computers how to bullshit. We've also managed to finally invent something that lets us communicate relatively directly with a computer using human languages. The language processing capabilities of an LLM are an astonishing multi-generational leap. These types of models will absolutely be the foundation for computing interfaces in the future. But they're still language models.

To me it feels like we've invented a new keyboard, and people are fascinated by the stories the thing produces.

"think" is doing a lot of heavy lifting there. Let's remember that neural networks are designed primarily not as an accurate brain models but as matrix models that can be calculated on in a feasible time scale on GPUs.