There was an article on HN a few weeks ago where someone detailed how they managed to get an old datacenter GPU to run in their consumer PC, getting decent performance with qwen. He spent something like $200 on the GPU (second hand of course).

So yeah, I think models on local hardware will be quite common soon among the tech savvy (such as people creating software).

▲

wrs an hour ago | parent [-]

Especially considering the millions of 2026-class data center GPUs that massively overinvested companies are currently buying, which will be obsolete in a few years.

▲

treis an hour ago | parent | next [-]

I think those are going to be run until they die. The capex vs opex is too high to obsolete them in a few years. They'll keep serving current gen LLMs for as long as they keep running.

	▲	Chu4eeno 29 minutes ago \| parent [-]
		They can also be used for other things than running the main frontier whatever model as well. E.g. grok isn't truly multi-modal, it has a callable tool that is a separate VLM it invokes on image URLs or files (for a long time it was grok-1.5v, but I think they have upgraded now, it was pretty bad). And then you have the small summarizer models for the CoT/thought traces, the guidable summarizer models for the standard browse tools, etc. There's a ton of stuff that can use an aging GPU.

▲

nok22kon 15 minutes ago | parent | prev [-]

H100 were released in Oct 2022. They are now more expensive than at release time.