It seems like there will be a collapse in building data centers, to the point that a large amount of the newly built ones will be reclaimed for other things. There is a very large group of people who are against building new data centers because they hate AI (and any pollution related arguments these people make are in support of their primary concern, being AI). But on the flip side of that, even people that I know that go out of their way to use AI, mostly seem interested in building their own AI rigs and running everything locally. I know that's what I do; I almost never use any hosted AI services these days, but I'm using more tokens than ever. The demand for data centers from people who actually like AI even seems to be plummeting.

▲

Gormo 2 hours ago | parent | next [-]

We're already at the point where a few months worth of an SMB's token usage from the SaaS LLM providers costs is equivalent to the cost of installing on-prem infrastructure capable of running cutting-edge open-weight models at scale. At my company, we recently installed a server with an array of Gaudi 2 cards on our server rack, and have set up an Open WebUI frontend to expose an LLM connected to all of our internal resources to our staff. The total cost was about $20k, which we'd easily eat up with six months worth of equivalent Claude usage.

We'd originally set this up to be able to locally run larger models, in the 300-400 billion parameter range, but the rate of improvement of open-weight models has been so fast that, coupled with extensive custom skill creation, we're now getting similar results out of Qwen3.8 27B to what we were getting out of Qwen 3.5 297B when we started out with the project, which frees enough memory to allow 20-25 users to have 256K context concurrently. Both the hardware, the software, and the models are improving at an accelerating rate.

Investing in data centers to support SaaS LLM providers today feels a bit like investing in mainframes and minicomputers in the late '70s, with a massive paradigm shift lurking right around the corner.

Actually, it's probably already closer to the early '80s, given that purpose-built local AI workstations are already available at price points lower than the inflation-adjusted initial price of the original IBM PC.

▲

GPerson an hour ago | parent | prev | next [-]

You don’t get to 25 or 30 trillion total addressable market by providing AI for the people who like AI. The whole plan is to replace all white collar work with AI. This requires more data centers. The people who benefit the most from this will see the least of the negative effects.

▲

nonethewiser 2 hours ago | parent | prev | next [-]

How is anti pollution argument pro ai?

And even if you do inference at home you are not training the models. Moreover, most people certainly are not doing inference locally.

▲

Zambyte 2 hours ago | parent [-]

> How is anti pollution argument pro ai?

What? (Edit: I think you misread "in support of their primary concern, being AI" as "in support of AI"? People who complain about pollution are usually mostly concerned about AI, and they use arguments of pollution to strengthen their argument against AI related things.)

> And even if you do inference at home you are not training the models.

No, and training does use a large amount of energy on a large amount of hardware. But:

- That sort of workload doesn't really require many distributed data centers, only a few powerful ones. I believe training is also getting cheaper for the achieving higher levels of capabilities, but I don't think the efficiency advancement has been as dramatic as it has been for inference.

- I believe we are really hitting a wall of diminishing returns, especially at the high end of large models. There is lower demand for training, because models are good enough to have a reasonably long shelf life at this point. Year+ old models that were SOTA in their time are still useful today. The demand for training is going down.

> Moreover, most people certainly are not doing inference locally.

Maybe not, but I think most people who go out of their way to use LLMs because they find them useful for their work actually are. Though most inference is probably from people who do it accidentally (as a part of a search result or something) or students who don't have the resources to do it themselves. But basically every software engineer that I personally know that uses AI for programming is either running their inference on their own hardware, or is talking about building a rig to do it.

	▲	bamboozled 2 hours ago \| parent [-]
		It’s wild to accuse people who are anti pollution of being anti ai, look how fucked the climate is, people don’t want to see it more fucked , is that hard to understand ?

▲

XorNot 2 hours ago | parent | prev [-]

What are you running at home, out of interest?

While I cautiously circle the AI programming concept, one thing close to my mind has definitely been whether the ultimately cloud-tied nature of it is what makes me much more stand offish to it.

▲

kodoman 2 hours ago | parent | next [-]

I am GPU poor and have a piece of crap rx570 8GB but still get ok performance out of qwen3.5-9B-4bit and various fine tuned versions. Plan on finally buying a decent gpu like a amd 7900 xtx. I still use claude for coding primarily but do research locally. What everyone has said about qwen3.8 though is making me feel I really need to invest in a decent rig.

▲

vrganj 2 hours ago | parent | prev [-]

Not the parent, but I'm enjoying Qwen 3.8 27B on my Strix Halo machine.

	▲	Zambyte 2 hours ago \| parent [-]
		Also running Qwen 3.8 27b, on a 7900 XTX. I've been able to comfortably do a large majority of my programming locally since about Qwen 3.5. I also use pi.dev as my agent harness, and ollama as my inference engine. Though I've been considering switching out the latter for something else, since the main advantage of ollama is the ease of switching models, which I don't really do often anymore.