Most of this might be true for LLM's but years of investing experience has created a mental model of looking for the tech or company that sucks and yet keeps growing.

People complained endlessly about the internet in the early to mid 90s, its slow, static, most sites had under construction signs on them, your phone modem would just randomly disconnect. The internet did suck in alot of ways and yet people kept using it.

Twitter sucked in the mid 2000s, we saw the fail whale weekly and yet people continued to use it for breaking news.

Electric cars sucked, no charging, low distance, expensive and yet no matter how much people complain about them they kept getting better.

Phones sucked, pre 3G was slow, there wasn't much you could use them for before app stores and the cameras were potato quality and yet people kept using them while they improved.

Always look for the technology that sucks and yet people keep using it because it provides value. LLM's aren't great at alot of tasks and yet no matter how much people complain about them, they keep getting used and keep improving through constant iteration.

LLM"s amy not be able to build software today, but they are 10x better than where they were in 2022 when we first started using chatgpt. Its pretty reasonable to assume in 5 years they will be able to do these types of development tasks.

At the same time, there have been expectations about many of these that did not meet reality at any point. Much of this is due to physical limitations that are not trivial to be overcome. Internet gets faster and more stable, but the metaverse taking over did not happen partially because many people still get nausea after a bit and no 10x scaling fixed that.

A lot of what you described as "sucked" were not seen as "sucking" at the time. Nobody complained about the phones being slow because nobody expected to use phones the way we do today. The internet was slow and less stable but nobody complained because they expected to stream 4k movies and they could not. This is anachronistic.

The fact that we can see how some things improved in X Y manner does not mean that LLMs will improve the way you think they will. Maybe we invent a different technology that does a better job. After it was not that dial up itself became faster and I don't think there were fanatics saying that dialup technology would give us 1Gbp speeds. The problem with AI is that because scaling up compute has provided breakthroughs, some think that somehow with scaling up compute and some technical tricks we can solve all the current problems. I don't think that anybody can say that we cannot invent a technology that can overcome these, but if LLMs is this technology that can just keep scaling has been under doubt. Last year or so there has been a lot of refinement and broadening of applications, but nothing like a breakthrough.

> Phones sucked, pre 3G was slow, there wasn't much you could use them for before app stores and the cameras were potato quality

This is a big rewrite of history. Phones took off because before mobile phones the only way to reach a person was to call when they were at home or their office. People were unreachable for timespans that now seem quaint. Texting brought this into async. The "potato" cameras were the advent of people always having a camera with them.

People using the Nokia 3210 were very much not anticipating when their phones would get good, they were already a killer app. That they improved was icing on the cake.

As others have mentioned you are just writing your own history to suit your narrative. There is no evidence to support "People complained endlessly about the internet in the early to mid 90s,".

In the early and 1990s, people effectively did not use the internet. Usage was tiny and miniscule, limited to only tiny niche groups. People heard about the internet via the 90 second blurb on the evening new show. It wasn't until sometime after the launch of Facebook that the internet was even mainstream. So I really don't think people complained about the internet being slow that they weren't using.

I can go on here, but I don't really need to spend paragraphs refuting something that is obviously false.

This is such selective hindsight, though. We remember the small minority of products that persisted and got better. We don't remember the majority of ones that fizzled out after the novelty wore off, or that ultimately plateaued.

Me, I agree with the author of the article. It's possible that the technology will eventually get there, but it doesn't seem to be there now. And I prefer to make decisions based on present-day reality instead of just assuming that the future I want is the future I'll get.

I'm not a fan of the argument that LLMs have gotten X times better in the past few years, so thusly they will continue to get X times better in the next few years. From what I can see, all the growth has mostly come from optimizing a few techniques, but I'm not convinced that we aren't going to get stuck in a local maxima (actually, I think that's the most likely outcome).

Specifically, to me the limitation of LLMs is discovering new knowledge and being able to reason about information they haven't seen before. LLMs still fail at things like counting the number of b's in the word blueberry or not getting distracted by inserting random cat facts in word problems (both issues I've seen appear in the last month)

I don't mean that to say they're a useless tool, I'm just not into the breathless hype.

> LLM"s amy not be able to build software today, but they are 10x better than where they were in 2022 when we first started using chatgpt. Its pretty reasonable to assume in 5 years they will be able to do these types of development tasks.

We can expect them to be better in 5 years, but your last assertion doesn't follow. We can't assert with any certainty that they will be able to specifically solve the problems laid out in the article. It might just not be a thing LLMs are good at, and we'll need new breakthroughs that may or may not appear.

We also thought 3D printing would print us a car, but alas.

FWIW - 3d printing has come a far way, and I personally have a 3D printer. But the idea that it was going to completely disrupt manufacturing is simply not true. There are known limitations (how the heck are you going to get a wood polymer squeezed through a metal tip?) and those limitations are physics, not technical ones.

People also complained a lot about VR.

And NFTs had a lot of loud detractors.

And everyone complained about a million other solutions that did not go anywhere.

Still, a bunch of investors made a lot of money on VR and very much so on NFT. Investments being good is not an indicator of anything being useful.

I see a bit of distinction here, that the foundation models aren't actually 10x better than in 2022. What's improved though is that we have far more domain knowledge of how to get more out of slightly improved models.

So consider your analogy, that the internet was always useful, but it was javascript that caused the actual titanic shift in the software industry. Even though the core internet backbone didn't radically improve as fast as you imagine it would have. Javascript was hacked together as a toy scripting language meant to make pages more interactive, but turns out, it was the key piece in unlocking that 10x value of the already existing internet.

Agents and the explosion of all these little context services are where I see the same thing happening here. Right now they are buggy, and mostly experimental toys. However, they are unlocking that 10x value.

those are really good points, but LLMs have really started to plateau off on their capabilities haven’t they? the improvements from gpt2 class models to 3 was much bigger then 3 to 4, which was only somewhat bigger than 4 to 5

most of the vibe shift I think I’ve seen in the past few months to using LLMs in the context of coding has been improvements in dataset curation and ux, not fundamentally better tech

> Twitter sucked [...] Electric cars sucked [...] Phones sucked

All these things are not black boxes and they are mostly deterministic. Based on the inputs, you EXACTLY know what to expect as output.

That's not the case with LLMs, how they are trained and how they work internally.

We certainly get a better understanding on how to adjust the inputs so we get a desired output. But that's far from guaranteed at the same level as the examples you mentioned.

That's a fundamental problem with LLMs. And you can see that in how industry actors are building solutions around that problem. Reasoning (chain-of-thought) is basically a band-aid to narrow a decision tree, because the LLM does not really "reason" about anything. And the results only get better with more training data. We literally have to brute-force useful results by throwing more compute and memory at the problem (and destroying the environment and climate by doing so).

The stagnation of recent model releases does not look good for this technology.

> Electric cars sucked, no charging, low distance, expensive and yet no matter how much people complain about them they kept getting better.

It takes them over a century to get to this current point.

Now think about hoverboards, self-cleaning shirts, moon bases, flying cars, functioning democracies, whatever VR tech is described in snow crash as well. Where on the spectrum will LLMs fall?

"it's pretty reasonable".. big jump?