Companies with billions of $, stopping R&D on the most revolutionizing technology/ breakthrough is something I can’t understand. I do understand there’s an AI Race to win, and that is with transformer and current LLMs architectures, but if research could find a better, more efficient approach they would re-shape this race, lead it, or even dominate it.

The problem is the "if". There is no guarantee that there's a "better, more efficient approach" to be found.

So far, the attempts to replace the transformer LLM with something else have yielded either "the same transformer LLM but with this one tweak that makes it 10% better" or "a radically different architecture that's a bitch to work with, and performs, best case, about the same as the transformer LLM".

By now, I'm firmly in the camp of "the architecture is good enough". Rifle is fine. Transformer is fine. Get better at the transformer before you look at anything else.

I think it’s more likely that some future LLM model finds a more efficient architecture than a human does, at this point. So yeah I’d go all in on LLMs too.

They are losing tons of money on it. Simple as.

Me neither. When revenues are growing every year, a fixed chunk of researchers' salaries (maybe growing a few % each year) doesn't really hurt, but R&D offers a nice upside.

Big corporations, by nature, are not rational in the decisions they make.

That's why they'll always be room for startups.

But is it a race to zero? Maybe AI will make very expensive things (eg software engineering) very cheap, but there will be enough competition that the margins are tiny and don't actually justify putting crazy dollar amounts into research