> you may run some models locally if only from a cost perspective

I have a hard time believing running a model on a laptop will be cheaper than running it in a datacenter. Why wouldn't economies of scale apply here as with every other computation?

▲

zozbot234 2 hours ago | parent | next [-]

The datacenter setting has huge economies of scale for low-latency, just-in-time inference using extremely large models, but that's not the only viable use of AI. Batched, unattended inference of possibly smaller and weaker models, while theoretically viable in a datacenter setting, is far from the best use of that hardware. This is where local AI is at its best.

▲

dgellow 5 hours ago | parent | prev | next [-]

A laptop is really a pretty bad form factor to run LLMs. Worst cooling, more expensive memory that you cannot replace, resell value depreciating fast. It’s fine for tinkering, small scale research, and demos but it’s definitely niche.

The vision NVIDIA is selling is pure marketing IMHO

▲

wazdra 4 hours ago | parent | prev | next [-]

This is assuming that you'll be priced the fraction of computing that you consumed. But you are actually paying for their infrastructure, for the R&D (and also the computation that went into training the model) etc. It is not clear that, for your own small computations, this kind of costs are needed, but you will still pay your share in the investment the provider made so that they could serve everyone's computation needs.

	▲	hungryhobbit 3 hours ago \| parent \| next [-]
		But, currently ... you're not. AI companies are operating at a loss, and are being subsidized by their investors. Local may or may not be cheaper than remote now, depending on the details, but the factors you describe won't affect the math nearly as much as they will once that subsidization ends.
	▲	wjnc 4 hours ago \| parent \| prev [-]
		In that analogy bigtech AI is currently investing in cleaner air for all of us? We _could_ breath it through their hose, but might as well breath it outside.

▲

itishappy 2 hours ago | parent | prev | next [-]

It's cheaper for the AI provider to use your laptop instead of their datacenter.

▲

jerf 3 hours ago | parent | prev | next [-]

What "every other computation"? I seem to have a lot processing power at my disposal here, between my cell phones, laptops, gaming PCs, various other hardware devices.

You're going to need to analyze the problem much more deeply because it sound like the standards you are implicitly applying would result in "economically, everything should be centrally hosted" but that is clearly not the result that obtains. Even a modern mid-grade cell phone is no slouch; you may not be running a current-gen frontier AI on it but you certainly can do a lot of other rather intense things locally that would have been laughable 10 years ago, like suprisingly high powered games.

▲

TylerE 2 hours ago | parent | prev [-]

Because economy of scale isn't really the right metric here. A machine you were you were going to buy anyway essentially has a TCO of $0.

	▲	dofm 32 minutes ago \| parent [-]
		AI models will pretty undeniably affect your electricity bill; yes you already own the computer, but it will cost more to run it if it's doing inference!