Three-year-old models aren't useful because there are (1) cheaper models that are roughly equivalent, and (2) better models.

If Sonnet 4.6 is actually "good enough" in some respects, maybe the models will just get cheaper along one branch, while they get better on a different branch.

I was just thinking earlier today how in an alternate universe, probably not too far removed from our own, Google has a monopoly on transformers and we are all stuck with a single GPT-3.5 level model, and Google has a GPT-4o model behind the scenes that it is terrified to release (but using heavily internally).

They didn't YOLO ChatGPT. There were more than a few iterations of GPT-3 over a few years which were actually overmoderated, then they released a research preview named ChatGPT (that was barely functional compared to modern standards) that got traction outside the tech community because it was free, and so the pivot ensued.

I also remember when the playstation 2 required an export control license because it's 1GFLOP of compute was considered dangerous

that was also brilliant marketing

In 2019 the technology was new and there was no 'counter' at that time. The average persons was not thinking about the presence and prevalence of ai in the way we do now.

It was kinda like a having muskets against indigenous tribes in the 14-1500s vs a machine gun against a modern city today. The machine gun is objectively better but has not kept up pace with the increase in defensive capability of a modern city with a modern police force.

That's rewriting history. What they said at the time:

> Nearly a year ago we wrote in the OpenAI Charter : “we expect that safety and security concerns will reduce our traditional publishing in the future, while increasing the importance of sharing safety, policy, and standards research,” and we see this current work as potentially representing the early beginnings of such concerns, which we expect may grow over time. This decision, as well as our discussion of it, is an experiment: while we are not sure that it is the right decision today, we believe that the AI community will eventually need to tackle the issue of publication norms in a thoughtful way in certain research areas. -- https://openai.com/index/better-language-models/

Then over the next few months they released increasingly large models, with the full model public in November 2019 https://openai.com/index/gpt-2-1-5b-release/ , well before ChatGPT.

Yeah, and Jurassic Park wouldn't have been a movie if they decided against breeding the dinosaurs.

And AI is currently killing a bunch of markets intentionally: the RAM deal for OpenAI wouldn't have gone through the way it did if it wasn't done in secret with anti-competitive restrictions.

There's a world of difference between what's happening and RAM prices if OAI and others were just bidding for produced modules as they released.

I mean enshittification has not begun quite yet. Everyone is still raising capital so current investors can pass the bag to the next set. Soon as the money runs out monetization will overtake valuation as top priority. Then suddenly when you ask any of these models “how do I make chocolate chip cookies?” you will get something like:

> You will need one cup King Arthur All Purpose white flour, one large brown Eggland’s Best egg (a good source of Omega-3 and healthy cholesterol), one cup of water (be sure to use your Pyrex brand measuring cup), half a cup of Toll House Milk Chocolate Chips…

> Combine the sugar and egg in your 3 quart KitchenAid Mixer and mix until…

All of this will contain links and AdSense looking ads. For $200/month they will limit it to in-house ads about their $500/month model.

Only until the music stops. Racing to give away the most stuff for free can only last so long. Eventually you run out of other people’s money.

Or self host the oss models on the second hand GPU and RAM that's left when the big labs implode

It's the low cost of labor in addition to lack of environmental regulation that made China a success story. I'm sure the competition helps too but it's not main driver

Consequence is they are now facing an issue of “cancer villages” where the soil and water are unbelievably poisonous in many places.