> Cloud providers are currently “un-differentiated”, but there are three huge ones making profits and some small ones too. Hosting is an economy-of-scale business and so is inference.

Anybody who has worked in a compliance heavy segment (PCI-DSS, HIPAA, etc.) will tell you the big 3 clouds have very significant differences from the smaller players. The differentiation is not on compute itself, but on the product. It's partially why products like AWS Bedrock exist and are actively placing model providers both in competition with eachother and AWS itself which is exactly the market dynamic they should seek to avoid.

> The idea that OAI is un-differentiated is just weird. They have a massively popular consumer offering, a huge bankroll, and can continue to innovate on features. Their consumer offering has remained sticky even though Claude and Gemini have both had periods of being the best model to those in the know.

This is exactly where this line of reasoning goes off the rails. The consumer market is problematic (see the recent post about the segment its growing in; basically young women of limited spend in low income countries); a huge bankroll is also a huge liability, model providers are on a clock to get huge or die, and the innovation we are seeing is effectively attempting to "scale-up" models, not provide novel features.

> Their consumer offering has remained sticky even though Claude and Gemini have both had periods of being the best model to those in the know.

This isn't a good thing with current market mix.

> And generally speaking there are huge opportunities to do enterprise integrations and build out the retooling of $10T of economic activities, just with the models we have now; a Salesforce play would be a natural pivot for them.

Do you have any indication these are achieving buy in or profitable? Most significantly, we have seen a recent study by MIT that 95% of generative AI pilots fail. The honeymoon period is rapidly coming to a close. Tangible results are necessary.

This only applies if you think one of two things; First, that it is guaranteed that this specific line of inquiry will lead to development of a form of superintelligence or otherwise broadly applicable development; or second, the form of machine learning technologies that unlocks or otherwise enables a market which would otherwise be inaccsesible that justifies this investment.

To date, no evidence of either even exists. See Zuckerbergs recent live demo of Facebooks Ray Bans technology, for example.

I think it's possible we are in datacenter GPU overcapacity already, and NVIDIA is burning its stock to avoid the music stopping.

It's already happening in China that datacenters are at GPU overcapacity. I wouldn't be surprised if it occurs here.

Currently, selling LLM inference is a red queen race: the moment you release a model, others begin distilling and attempting to sell your model cheaper, avoiding the expensive capitalized costs associated with R&D. This can occur because the LLM market is fundamentally -- at best -- minimally differentiated; consumers are willing to switch between vendors ("big labs", as you call them, but they aren't really research labs) to whomever offers the best model at the lowest price. This is emphasized by the distributors of many LLMs, developer tools, offering ways to switch the LLM at runtime (see https://www.jetbrains.com/help/ai-assistant/use-custom-model... or https://code.visualstudio.com/docs/copilot/customization/lan... for an example of this). The distributors of LLMs actively working against LLM providers margin provides an exceptionally strong headwind.

This market dynamic begets a low margin race to the bottom, where no party appears able to secure the highly attractive (think the >70% service margin we see in typical tech) unit economics typical of tech.

Inference is a very tough business. It is my opinion (and likely the opinion of many others) that the margins will not sustain a typical "tech" business without continual investment to attempt to develop increasingly complex and expensive models, which itself is unprofitable.