| ▲ | silverFork 2 hours ago | |
I said in my opinion.. But here a reference; https://www.tomshardware.com/pc-components/gpus/datacenter-g... | ||
| ▲ | JumpCrisscross an hour ago | parent | next [-] | |
> I said in my opinion I've been pitched data-center deals. They depreciate on an 18- to 24-month schedule, well under the Tom's Hardware terms. The ones who went online a year or two ago aren't losing their chips like ducklings through a storm gate; if anything, their resale value has remained remarkably stable because compute production is the bottleneck. You've given a source (a great one, btw) for depreciation but not revenue. If you can name a company, I can look if I have a public source that confirms what I know. But broadly speaking, no, unit economics in the AI economy is weirdly sound, though I suspect it's because every non-AI CEO is blowing out their budgets on frivolous spending. | ||
| ▲ | phil21 an hour ago | parent | prev [-] | |
> The utilization rates of GPUs for AI workloads in a datacenter run by cloud service providers (CSP) is between 60% and 70%. With such utilization rates, a GPU will typically survive between one and two years, three years at the most, according to a quote allegedly made by a principal generative AI architect from Alphabet and reported by @techfund, a long-term tech investor with good sources. This is just extremely unbelievable to me. I am certainly not operating at a level the hyperscalers are and have much more limited direct experience. But I do actually put various GPUs inside datacenters (and much harsher locations) and have operated them at balls-to-the-wall 100% utilization for over a decade now. You get the typical bathtub curve of failures. Unless the hyperscalers are operating these things even more overclocked and beyond thermal specification limits than early GPU crypto miners used to do, I simply cannot believe that the average hardware life is less than the useful life of the whole chip generation itself. I have plenty of decade old GPUs that operate today just fine. Both consumer and datacenter form factors. The failures tend to be board-level like capacitors and such, so if you are operating at a massive scale partnering with someone who can fix those relatively cheaply is not all that difficult either. It could be that these H200 and above class sort of stuff is engineered extremely fragile, but I seriously doubt it. The prevailing "common knowledge" pre-AI for GPUs were that they'd burn out in a year or two of heavy use, and that was simply untrue. I saved a ton of money buying batches of used units because everyone was terrified of this - and had no more early failures than I did buying brand new after basic refurb of re-pasting and putting a new fan on them. | ||