Remix.run Logo
▲ blurbleblurble a day ago

You sound exactly like the people who didn't understand personal computers because mainframes existed.

▲bunderbunder a day ago | parent | next [-]

I don't see how. I'm not saying that consumer-grade hardware isn't useful because enterprise-grade hardware exists. I'm saying that enterprise-grade hardware isn't particularly practical for most consumers. Personal computers were many things, but what they generally weren't was people installing IBM S/390s in their living rooms. Because some of the same design features that make them great for centralized, high volume business workloads also makes them kind of a PITA if that isn't what you need.

For my part, I do work on machine learning projects at home. But I'm nowhere near operating at a scale where datacenter GPU features like being optimized for 4- and 8-bit calculations would be a net benefit. Nor do I have any particular desire to have one of those things making my home office sound like I live next to a steel mill.

It's not too far off from why I don't have a box truck parked in my garage even though it's technically more capable than my Kia. And on the occasions when that is what I need, the local hardware store will gladly rent one to me for $20 per day.

▲blurbleblurble a day ago | parent [-]

They're buying out RAM and consumer GPUs too.

▲Ekaros a day ago | parent | prev | next [-]

Mainframes did not make and still don't make very good personal computers. Same goes for these "gpus". They are the modern day mainframes. (I know we still have mainframes, but that is besides analog)

▲fasterik a day ago | parent | prev [-]

What exactly is the average consumer supposed to do with an NVIDIA H100 or GB200?

▲PowerElectronix a day ago | parent [-]

They can have a pretty good LLM running in there, or cad design. For the average joe, having the free tier of AI without the breach of privacy is pretty good already.

▲fasterik a day ago | parent [-]

For the average joe, spending $10k on the server-grade hardware needed to run the GPU, plus $100-$200 per month for power is not a great investment.

▲iamnothere a day ago | parent | next [-]

The hope is that a crash would lead to fire sale prices, and as for power, you may not need to run your system 24x7 depending on your use case.

▲PowerElectronix a day ago | parent | prev [-]

As the other comment answered, if a crash materializes like the ones before (dotcom bubble, crypto crash, etc) you'd be able to get a lor of ram and a couple of high end GPUs (5090, 4090 and the like) for a grand or two. With that you can a pretty beefy model locally.