I have two 128gb Strix Halos and have been extremely excited about Antirez's (Redis author) work on DS4, especially with 4bit quant using two machines: https://github.com/antirez/ds4

Right now the speed isn't good for GLM 5.2, Deepseek V4 Flash speed is okay for me (actually reading the output) and quite usable. See kyuz0's great recent video here: https://www.youtube.com/watch?v=PkKXm_mKCCM

With a bit more speed and model improvements, local AI becomes a reasonable practical thing! The biggest problem is all the tech companies making consumer hardware completely unaffordable, and I don't think this is accidental. Look at Micron's profits and share price lately...

I got my Strix machines for ~2k eur each, best computers this 90s kid has ever owned, but those days are gone :(

▲

sspiff 2 hours ago | parent | next [-]

I had a Strix Halo laptop with 128GB which unfortunately died last week. I paid 2800 euro for it. If I buy the same machine today, the sticker price is 7899.

The device was not perfect by any means, but the ability to run fairly large models is some kind of magic.

	▲	barbacoa an hour ago \| parent [-]
		>sticker price is 7899. It's not even worth it at that point. You can get a used enterprise grade SXM baseboard with 4-8 V100/A100 GPUs off eBay at a similar price. That will even get you actual HMB ram and NVlink. Along with 10x the AI performance, assuming you don't care about your electricity bill of course.

▲

Gareth321 2 hours ago | parent | prev | next [-]

I was hoping to buy a competent local model machine later this year but given the prices I’m shelving that for now. Especially because the frontier models are very cheap relative to the cost of building my own setup. Especially because AI specialised hardware and processors are improving very fast, meaning hardware we buy now will become obsolete for this use case much faster than for traditional computer use cases.

In 1-3 years the hardware crunch will be over, local distilled models will provide Opus 4.8 like intelligence, and the hardware will exist to provide usable performance.

▲

rjzzleep 3 hours ago | parent | prev | next [-]

Last year you could buy a AI Max 395+ with 128G for 2.5k, now it's almost $4k.

Or maybe you're right, I originally remembered 2k as well. I wanted to wait for the AI Max 395+ upgrade of my laptop, and now it makes no sense to upgrade.

▲

stymaar 3 hours ago | parent | next [-]

> Last year you could buy a AI Max 395+ with 128G for 2.5k, now it's almost $4

Only if you pay the Framework premium.

https://www.bosgamepc.com/products/bosgame-m5-ai-mini-deskto...

I don't have access to the USD price, but it's 2500€ (tax included), up from 1600€ in November when I ordered mine.

▲

pixelpoet 3 hours ago | parent | prev | next [-]

I think people buying laptops for AI use are, sorry, just plain crazy. You overpay for the screen and keyboard and battery and whatever, plus you get much worse thermal performance because of basic physics (area vs volume). My Framework Desktop has a Noctua cooler which works really well.

[Tangent: all my life I've been downvoted into a smoking hole in the ground, particularly on reddit r/hardware, for questioning the wisdom of laptops for high performance computing, including gaming. Everyone insists they need the mobility, and then just leave it plugged in the whole time, absolutely refusing to admit it's about aesthetic preference.]

▲

kamranjon 2 hours ago | parent | next [-]

I generally agree for everything except Macbook Pros which outperform most available desktop setups for AI tasks - but they are also now out of reach for most people after the price hikes (6.7k now for 128gb, i got mine for 4.7k just about a year ago).

Honestly I think this is just a bad time to be buying hardware - everything is marked up an insane amount that doesn't really make sense.

▲

rzzzt 2 hours ago | parent | prev | next [-]

For me the smaller footprint, lower power consumption and portability (admittedly between desks only) are the three advantages of using a laptop over a desktop for these purposes.

▲

pixelpoet an hour ago | parent [-]

The Strix Halo mini PCs use the exact same chip, and have a much smaller footprint than any laptop. Have you seen the size of these machines? I can and have easily popped my daily driver computer into my very small backpack to attend a demoparty for example.

With the laptop you probably won't get silent operation at the peak 100-140w, i.e. you've now massively overpaid for lower performance.

▲

throwa356262 an hour ago | parent [-]

Can you get these from vendors like Asus and lenovo these days?

The ones I've seen on aliexpress are from unknown Chinese vendors.

	▲	pixelpoet 34 minutes ago \| parent [-]
		I have a Framework Desktop as primary PC (great cooling, beautiful case with handle) and the Bosgame M5 dedicated for AI use. I was also a bit wary about Bosgame but TBH they've been great and the machine is rock solid, if a little noisier than and not as pretty as the FD. You can just buy from them directly and be fine, best computer deal out there by a mile.

▲

Gareth321 2 hours ago | parent | prev [-]

I’m mostly with you but there are some people who like to use one machine for both laptop and AI work, and it’s much cheaper than buying two separate devices.

▲

Tepix 2 hours ago | parent | prev [-]

The cheapest ones with 128GB were 1580€/$1840 as late as mid December.

▲

rnewme 6 hours ago | parent | prev | next [-]

What's the advantage of ds4 over llama.cpp, esp if down the line they upstream his forked kernels?

▲

pixelpoet 3 hours ago | parent | next [-]

IIRC llama.cpp doesn't implement DSv4's compressed attention mechanism, and while it does use (credited) parts of llama.cpp, it's focused on this great model for now. Much of this is covered better in the repo's readme.

	▲	rnewme 16 minutes ago \| parent [-]
		In repo Readme and antirez reddit comments there was also expressed willingness to upstream.

▲

francisduvivier 3 hours ago | parent | prev [-]

I think mainly that he can move much faster with specific improvements targeting Deepseek on Systems with unified memory (Mac or Strix). It's a lot easier to optimize if you don't need to worry about all the other architectures. So optimize he did and it's just a lot faster than llama cpp for deepseek v4 pro and flash. Also interesting features are more doable, like SSD streaming, which makes it possible to load MOE weights for a model larger than your VRAM, I don't see that landing in llama cpp anytime soon.

▲

gruez 6 hours ago | parent | prev [-]

>The biggest problem is all the tech companies making consumer hardware completely unaffordable, and I don't think this is accidental. Look at Micron's profits and share price lately...

You realize "tech companies" isn't a monolith? Micron charging inflated prices doesn't magically benefit OpenAI. The "high prices keep out competitors" theory doesn't make much sense either. It's like saying Dennys benefits from higher egg prices because it makes cooking eggs at home more expensive.

	▲	sdf4j 5 hours ago \| parent \| next [-]
		You got it wrong. Use appliances instead of eggs. If getting an oven gets more expensive I rather keep going to Dennys. It’s classic capex vs opex. I’d keep paying my openai subscription instead of dropping $3k to run a subpar model. If the thing costs $1k I would consider it.
	▲	mkj 5 hours ago \| parent \| prev [-]
		openai etc are going to have a higher utilisation of the hardware so can afford it more than small companies/people. Efficient resource use matters more when they're expensive.