I think it still makes sense to wait. Hardware is currently hyper expensive and cloud models are subsidized. Waiting 2 years or so once memory prices have dropped and datacenters start wanting a profit would get you a usable setup that's more economical.

How much electricity does running your local models take?

(I'm not one of the people you're speaking of with a 128gb M5 but) if you want to run one of the medium-sized open-weights models (Qwen 27b, 35b, Gemma 4 26b, 31b) or larger, you get into an interesting optimisation space.

* yes, you can run it on an older/smaller GPU plus system RAM but performance will suffer

* if you want optimal GPU performance you need the model in VRAM plus context, so 24GB (3090, 4090) or 32GB (5090) cards, plus a system that's reasonable powerful to plug them in to. Ideally you'd have a multiple cards working together but for optimal performance this means either 2x 3090 or nvidia's workstation cards.

* you can go for a 128gb Strix Halo system, but the memory bandwidth isn't great and they're becoming increasingly more expensive (5.5k EUR for HP laptop, 3.9k EUR for GMKtec EVO-X2 mini PC)

* you can go for a 128gb DGX Spark (5k EUR+) which also has unspectacular memory bandwidth or RTX Spark (price unclear but probably not cheaper)

* or go for a Mac with a decent CPU and a good amount of RAM (bandwidth varies by model, but typically a bit better than Strix Halo/DGX Spark and worse than bespoke GPUs.

As usual with such questions, there are of course cheaper paths (if you want to accept the tradeoffs) but Macs are reasonable vs. competition for these workloads.

I don't even travel a ton but portability is huge. It's not a flex, it's a functional thing that lets me move around within my house or work while I'm at my parents or traveling or anywhere else. Other than my media collection that lives on my home server, I want most of my files to come with me on my laptop.

A mac with a boatload of RAM can run models that will exceed the limits of any GPU not worth at least twice the Apple hardware itself.

You get fewer tokens per second, but at some point the balance between quality and quantity makes the large model size worth the spend.

When you're spending this kind of money, you may as well treat yourself to a pretty screen and some decent speakers. Nothing the competition doesn't offer these days, but you get them for free with the car-priced RAM upgrade so why go for less.

I have a bunch of computers and gadgets, why settle on one?

I think it is because desktop computers with GPUs with enough VRAM to run interesting models are insanely expensive, hard to source and consume a lot of electricity and dissipate a lot of heat.

Yeah, it's a much better idea to buy many used 3090s. 4090s or 5090s if you can afford it. Way faster.

What GPU can I buy with >100GB of memory?

> will likely be resale-able for what I paid for it for the next 7-10 years

There is not a period in the history of computing where this is true of consumer hardware over a decade for anything other than hardware already at the very bottom of its depreciation curve. It is surprising to me that you state that as an obvious assumption.

I suppose if your base case is Taiwan war that may be true, but there's a lot of folks who seem to be assuming the current hardware crunch will go on indefinitely when the natural state of hardware is getting cheaper over time.

That makes a lot of sense! I have no idea how I'd use that much money, so maybe the 128gb MBP for messing around with local LLMs wouldn't sound so absurd :)