There's a big issue with a lot of thinking about these valuations, which is that LLM inference does not need the 5-nines of uptime guarantees that cloud datacenters provide. You are going to see small business investors around the world pursue the following model:

- Buy an old warehouse and a bunch of GPUs

- Hire your local tech dude to set up the machines and install some open-source LLMs

- Connect your machines to a routing service that matches customers who want LLM inference with providers

If the service goes down for a day, the owner just loses a day's worth of income, nobody else cares (it's not like customers are going to be screaming at you to find their data). This kind of passive, turn-key business is a dream for many investors. Comparable passive investments like car washes, real estate, laundromats, self-storage, etc are messier.

I use OpenAI's batch mode for about 80% of my AI work at the moment, and one of the upsides is it reduces the frantic side of my AI work. When the response is immediate I feel like I can't catch a break.

I think once the sheen of Microsoft Copilot and the like wear off and people realise LLMs are really good at creating deterministic tools but not very good at being one, not only will the volume of LLM usage decline, but the urgency will too.

Isn't this the whole premise of existing companies like SF Compute? [0]

[0] https://sfcompute.com/