Why? It's hard to see what they're good for outside of AI data center workloads. Your average member of the general public arguably doesn't have much use for a "graphics processing unit" that isn't actually capable of processing graphics

▲

Toynbeeidea 5 hours ago | parent | next [-]

I run free local models on my A4500. I wouldn't mind a little upgrade... if I could afford it.

▲

blurbleblurble 5 hours ago | parent | prev [-]

You sound exactly like the people who didn't understand personal computers because mainframes existed.

▲

bunderbunder 5 hours ago | parent | next [-]

I don't see how. I'm not saying that consumer-grade hardware isn't useful because enterprise-grade hardware exists. I'm saying that enterprise-grade hardware isn't particularly practical for most consumers. Personal computers were many things, but what they generally weren't was people installing IBM S/390s in their living rooms. Because some of the same design features that make them great for centralized, high volume business workloads also makes them kind of a PITA if that isn't what you need.

For my part, I do work on machine learning projects at home. But I'm nowhere near operating at a scale where datacenter GPU features like being optimized for 4- and 8-bit calculations would be a net benefit. Nor do I have any particular desire to have one of those things making my home office sound like I live next to a steel mill.

It's not too far off from why I don't have a box truck parked in my garage even though it's technically more capable than my Kia. And on the occasions when that is what I need, the local hardware store will gladly rent one to me for $20 per day.

	▲	blurbleblurble 4 hours ago \| parent [-]
		They're buying out RAM and consumer GPUs too.

▲

Ekaros 4 hours ago | parent | prev | next [-]

Mainframes did not make and still don't make very good personal computers. Same goes for these "gpus". They are the modern day mainframes. (I know we still have mainframes, but that is besides analog)

▲

fasterik 5 hours ago | parent | prev [-]

What exactly is the average consumer supposed to do with an NVIDIA H100 or GB200?

▲

PowerElectronix 5 hours ago | parent [-]

They can have a pretty good LLM running in there, or cad design. For the average joe, having the free tier of AI without the breach of privacy is pretty good already.

▲

fasterik 5 hours ago | parent [-]

For the average joe, spending $10k on the server-grade hardware needed to run the GPU, plus $100-$200 per month for power is not a great investment.

	▲	iamnothere 3 hours ago \| parent [-]
		The hope is that a crash would lead to fire sale prices, and as for power, you may not need to run your system 24x7 depending on your use case.