Remix.run Logo
▲ AngryData 7 hours ago

Im not convinced by the claims of infrastructure only being expensive to build initially. Hardware doesn't last forever especially in a data center, maintenance isn't free, electricity is not going to get significantly cheaper. If models get more efficient we won't need super expensive and expansive hardware build outs, if models keep getting more powerful and needy and are hardware limited, each new hardware generation will stomp over all this expensive and soon to be old hardware in electrical efficiency requiring another massive round of funding that players already holding debt will struggle to afford.

▲genxy 6 hours ago | parent | next [-]

If new hardware comes online that doesn't allow you to payback your billions in hardware, what then? The level of rigor in the article is low.

If we steel-they the point in the essay, it still doesn't matter because a bubble is setup in the economics of the system, not whether the underlying tech works and/or is profitable. It can both work, be profitable, but have been overcapitalized before the knock on effects are in-place in the rest of the economy. This is a crazy analogy, but what if 1700s earth figured out, theoretically how to build a fusion reactor that could supply all the energy needs of the whole planet. But it cost in their equivalent dollars, 1 quadrillion dollars? You can't even integrate under their money curve to scrape together that many dollars. The only economy you can make about that is in the research side to push the cost down. Then when its dialed in, you can launch that rocket AND the rest of the system has had time at many different levels to integrate AI at various levels. OpenAI dry barebacked its way into the economy and literally screwed everything up. If they really were an altruistic AI safety lab, they should have kept working on ChatGPT, but demoed it to the world including government, so we could as a planet decide where we are taking this.

It is a huge bubble, AI is over capitalized, we have no idea what the knock on effects are and really the only benefit right now is in programming, which is huge and civilization changing. But it also has massive downsides in that it has given the most powerful even longer levers to move the earth.

▲KaiserPro 42 minutes ago | parent | prev | next [-]

I just don't think the author really has any clue.

they seem to think that debt markets are infinite. "There's Still Plenty Of Capital To Raise"

> The ones that have issued public debt, like Alphabet and Meta, have excellent bond ratings. If they need more capital, they won't struggle to find people to give them a loan.

Yes, and that bond rating is based on the idea that they really don't have that much debt, and have a net profit of something like 30%. If they hoover up a load of debt, the rating will plunge.

> I don't think the rhetoric about the AI bubble is really about finances

Its exactly about finances. The market participants that are not bollocks deep in fake Private Equity valuations are worried about circular debt obligations that are off books. The SEC has been rather lenient about reporting of non-traditional obligations (hence why spaceX was allow to do what ever the fuck it did)

For example anthropic has about 450 billion in non-cancellable obligations. the S-1 puts it at 895 billion. all of that is someone's debt.

If openAI are doing the same thing, thats pretty close to 1.6trillion in debt, both for the same customers, at the time when customers are going "hmm this isn't giving us what we hoped for the price"

▲matwood 2 hours ago | parent | prev | next [-]

> electricity is not going to get significantly cheaper

The DC push may be the thing that finally gets renewables over the finish line in the US. For example, TX is around 35% renewables right now. If that continues to rise, we may see power fall in price. It’s all about incentives.

▲siod 2 hours ago | parent [-]

Without a requirement for DC's to add more power to the grid I don't think the price is going down, they're essentially adding to demand and the supply isn't particularly elastic. That being said this is a good opportunity for renewables, they're the cheapest form of power.

▲matwood an hour ago | parent [-]

And we just saw a pretty strong repudiation of that type of DC buildout. I have a little hope politicians can communicate it’s not DCs that are the issue, but the externalization of all the costs.

▲Tuna-Fish 6 hours ago | parent | prev | next [-]

So much this.

The current-gen hardware is expected to have only 2-3 years of useful life. New tech, and especially new architecture is going to make inference much faster and more efficient in the near future, and the big vendors are going to be more limited by datacenter floor space and power/cooling, so that the hardware will be replaced very quickly by better models.

▲ianm218 6 hours ago | parent | next [-]

For the time being this thesis hasn’t worked out though. H100’s are only going up in price and becoming more scarce [1]. At what point would you update if now going on 3 years into H100 lifecycle this doesn’t seem close to happen.

▲dofm 6 hours ago | parent [-]

But they are becoming more scarce while staying on Nvidia’s shelves buy-and-hold, or being leased back in some cases. They are not being installed by customers at anything like the rate the sales figures suggest. So yeah, these H100s will last a long time in the packaging.

Meanwhile, Meta is selling their spare AI data centre capacity.

▲gizajob 2 hours ago | parent | next [-]

Where are all these unused H100s sitting and where are the people who have been fired for buying them?

▲wombatpm 5 hours ago | parent | prev [-]

Isn’t that what SpaceX is doing too? Selling spare compute while they work on their rocket side hustle.

▲siod an hour ago | parent | prev | next [-]

People are buying v100s on ebay to run LLMs and Chinese manufacturers make custom PCIE boards for them. Older cards might not be useful for training but they're perfectly capable of doing inference work.

▲madaxe_again 4 hours ago | parent | prev | next [-]

Jevons' Paradox. If it becomes more efficient, cost goes down, demand goes up, use rises, revenue rises.

▲genxy 6 hours ago | parent | prev | next [-]

Look at DeepSeek! They have done full system optimization on their model, architecting it so that it would be cheap to serve. I am proud to call DeepSeek a world class tech engineering firm in a way that 99% of the tech world is not. They are a 1% company.

▲anamax 3 hours ago | parent | prev | next [-]

> The current-gen hardware is expected to have only 2-3 years of useful life.

Expected by whom and why?

Google hasn't retired any of its now ten-year-old TPUs, or any of the subsequent generations. Maybe TPUs are special or Google doesn't understand compute economics but either argument requires data.

If your argument is depreciation, you're confusing a tax concept (depreciation) with useful life. Yes, they're supposedly related, but in practice, the relationship is rather loose. (Residential housing depreciates in 28 years yet typically lasts 2-3x as long and 4x is not rare.) Google's V1 TPUs

▲zobzu 4 hours ago | parent | prev [-]

no, it's 10y. you just repurpose.

also, power and cooling agreements are usually 10y too.

▲coef2 6 hours ago | parent | prev [-]

It hardly seems to be a one time expense. GPUs and networking equipment may have a useful lifespan of only about five years before needing replacement, and it could be expensive to do so.