Remix.run Logo
stuxnet79 16 hours ago

Ok so Samsung, SK Hynix and Micron do not have the capacity to meet demand. Also, what little capacity they do have they are allocating to HBM over DRAM. Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years.

It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all the dry powder out of the private markets. Folks are now starting to ask difficult questions about their burn rate and revenue. It is increasingly looking like they might not commit to the purchase order they made which kick-started this whole panic over RAM.

Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?

bombcar 3 hours ago | parent | next [-]

Don’t the memory makers always get left holding the bag? I feel this has happened at least three times before.

formerly_proven 2 hours ago | parent | next [-]

DRAM and to a lesser degree storage are notorious for their feast and famine cycles

(Well that and collusion)

bombcar an hour ago | parent [-]

I don’t know if they realize that collusion lends itself to feast/famine.

whatever1 44 minutes ago | parent | prev [-]

All of the capital intensive businesses face this issue. Chemicals, Shipping, Semiconductors etc.

You get market signals that the demand is there, you acquire the necessary capital, you spend 5 years to build capacity, but guess what, 5 other market players did the same thing. So now you are doomed, because the market is flooded and you have low cash flow since you need to drop prices to compete for pennies.

Now you cannot find capital, you don't invest, but guess what, neither your competitors did. So now the demand is higher than the supply. Your price per unit sold skyrocketed, but you don't have enough capacity!

Rinse and repeat.

Capitalists claim that this is optimal.

lsiq 23 minutes ago | parent | next [-]

The book Capital Returns: Investing Through the Capital Cycle details this phenomenon, including historical cases.

If anything, it shows it's possible for you to arbitrage this and in doing so help "smooth out the cycle."

kev009 30 minutes ago | parent | prev | next [-]

Is the DRAM industry really capitalist? Focusing on just the Korean parties, it functions like a command economy. I would say the same about most high end semi-conductor manufacturing, TSMC, Intel, ASML are being commanded and driven by nation-state level decision making. Right now the command is to focus on high wattage centralized AI systems at the expense of everything else.

wordpad 31 minutes ago | parent | prev | next [-]

>Capitalists claim that this is optimal.

It's more optimal than planned economies until we have AI planned economies with realtime feedback, I guess.

Consumers get cheap goods during oversupply and most inefficient companies get elliminated during bust while consolidation leads to economies of scale.

whatever1 7 minutes ago | parent [-]

No this is literally a sign of an unstable system with too high of a gain K.

There is an alternative where legislation dampens this behavior but the short term profits will be lower. Hence the hawks don’t like it.

tgma 34 minutes ago | parent | prev [-]

> Capitalists claim that this is optimal.

Because that does not happen exactly as you say for all players. The demand signals will be processed and long-term risk is balanced against short-term gain in a distributed fashion, so not everyone will do the same.

Macha 19 minutes ago | parent | prev | next [-]

> Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?

The memory makers specifically did not scale up capacity to avoid being left holding the bag.

torginus 14 hours ago | parent | prev | next [-]

The Radeon VII came out in 2019 as a $700 consumer GPU with an 1TB/s HBM2 memory subsystem which is more than any consumer GPU you can get today, including the high-end ones afaik. At that point in time, there was a whole lineup of AMD GPUs with HBM going down into the midrange.

If they could make this stuff and sell it to regular people a decade ago for very palatable prices, why do they come up with the idea that this is the technology of the gods, unaffordable by mere mortals?

HerbManic 3 hours ago | parent | next [-]

I have been wondering this recently. It was the convention that if you wanted to keep costs down, try to keep the memory bus size down as low as possible. Still remember the awful Radeon 9200 SE - 64bit data bus that strangled an already slow GPU.

Heck, I have a phone with a 16bit memory bus for instance. The high(ish) clock rate only makes up the difference slightly.

But with general prices on all components going up, it might not be such a big factor any more.

HBM migght make sense for higher end products which can free up space for the lower end that will never use the tech.

tpm 8 hours ago | parent | prev | next [-]

> 1TB/s HBM2 memory subsystem which is more than any consumer GPU you can get today

5090 has 1.8 TB/s?

platevoltage 3 hours ago | parent | prev | next [-]

I was gonna say, I still use an AMD Vega that uses HBM2.

cco 14 hours ago | parent | prev [-]

That card only had 16GB of memory; its memory bandwidth was 1TB/s.

mrbuttons454 3 hours ago | parent | next [-]

The Pro variant had 32GB, I had one in a 2019 Mac Pro

imtringued 13 hours ago | parent | prev [-]

You're saying this in a world where AMD's highest end consumer GPU in 2026 is also limited to 16 GB.

thehamkercat 10 hours ago | parent [-]

RX7900 XTX has 24GB

fl4regun 26 minutes ago | parent [-]

this card is 4 years old, it's not on store shelves anymore.

Cthulhu_ 15 hours ago | parent | prev | next [-]

To add a more local hurdle as well, the Dutch power grid is at capacity and its managing company is now telling companies that planned to build a datacenter that they can't be connected to the grid until 2030, even though said companies already paid for and got guarantees about that connection.

That is, memory capacity is reserved for datacenters yet to be built, but this will do weird things if said datacenter construction is postponed or cancelled altogether.

consp 14 hours ago | parent | next [-]

That guarantee is not as much of a guarantee as stated in the media. You get a guarantee it will be planned at a certain time (as in looked at), not that it will be build. The cost of doing business is taking risks and mitigating them. There is a reason the nuclear plant in Borsele was build: an aluminium smelter. Maybe you should arrange for something similar as a datacenter (no politician will fall on a sword for that but you can try). The (original) power draw is about the same 80-100MW.

AnthonyMouse 3 hours ago | parent | prev [-]

> the Dutch power grid is at capacity and its managing company is now telling companies that planned to build a datacenter that they can't be connected to the grid until 2030, even though said companies already paid for and got guarantees about that connection.

Are the Netherlands a large proportion of global datacenters?

toast0 3 hours ago | parent | next [-]

Amsterdam hosts a major internet exchange. It's not a bad place to build a datacenter and there are many. Northern latitude brings free air cooling, but also additional distance to clients. Lots of peers in AMS-IX, but not a lot of oceananic cable landings (one with two paths to the US, but most of the submarine cables land nearby in Europe)

AnthonyMouse 2 hours ago | parent [-]

Whether it's generally a reasonable place to build them isn't the percentage. The number seems to be ~3%.

Dracophoenix 3 hours ago | parent | prev | next [-]

Yes. Amsterdam has one of the largest IXPs (AMS-IX) in Europe and is also one of the largest European markets for Internet Infrastructure services (i.e. hosting, DNS provision, domain name registration, etc.)

trvz 2 hours ago | parent [-]

And all of these are practically irrelevant for AI data centers.

estimator7292 3 hours ago | parent | prev [-]

Is that relevant? The grid in every country is getting ridiculously stressed by datacenters.

AnthonyMouse 2 hours ago | parent | next [-]

What the grid looks like in different countries is very different. The Dutch power grid is already almost 50% renewables, which is an inconvenience for adding capacity because that's around where you have to start really dealing with storage in order to add more.

In most other places the percentage is significantly less than that and then you can easily add more of the cheap-but-intermittent stuff because a cloudy day only requires you to make up a 10% shortfall instead of a 50% one, which existing hydro or natural gas plants can handle without new storage when there are more of them to begin with.

nslsm 2 hours ago | parent | prev [-]

>The grid in every country is getting ridiculously stressed by datacenters.

In every country? Citation needed.

danishanish an hour ago | parent | prev | next [-]

I think I’m missing something. Financially, what bag would the memory makers be holding here? I don’t think I’m well informed regarding how these deals were structured.

michaelbuckbee an hour ago | parent [-]

Memory makers make capital investements (build different factories, convert physical production lines, etc.) to meet orders that have been place for the next ~5 years.

OpenAI (or whoever) crashes and can't pay for the order leaving the memory makers in a tough spot.

xbmcuser 14 hours ago | parent | prev | next [-]

I am betting the pendulum swings faster to the other side to excess capacity as all the construction lies of Altman fall through with financiers waking up the the fact they can't build the infrasctructure as fast nor make any profits on that infrastructure that will get built.

trvz 2 hours ago | parent [-]

Those financiers can’t risk not being involved in a company with even just a slight potential for AGI.

zozbot234 2 hours ago | parent | prev | next [-]

There's actually plenty of demand for LPDDR even in the AI datacenter, because HBM is quite wasteful of area for any given memory capacity.

BeeOnRope 41 minutes ago | parent [-]

Wafer area?

nostrademons 3 hours ago | parent | prev | next [-]

What kind of consumer electronics can you build with HBM? That's the startup you should be founding...

daemonologist 2 hours ago | parent | next [-]

AMD has built some consumer GPUs in the recent past with HBM - RX Vega and Radeon VII (although I assume not all "HBM" is created equal).

hypercube33 33 minutes ago | parent | next [-]

Isn't their APU also capable of doing HBM? There was an Intel AMD hybrid chip that used unified a while back too.

christkv 2 hours ago | parent | prev [-]

My vega 56 still has 400gb/s of memory which is still insane for how old the card is.

zdw 2 hours ago | parent [-]

AMD's Hawaii architecture had 320GB/s on a 512b GDDR5 bus in 2013.

The Fiji XT architecture after it had 512GB/S on a 4096b HBM bus in 2015.

The Vega architecture did have 400GB/s or so in 2017, which was a bit of a downgrade.

CamperBob2 an hour ago | parent | prev | next [-]

HBM is just normal DDR RAM that's been packaged with (much) wider-than-usual data buses. That's where the high bandwidth comes from, not from high clock rates or any other innovation or improvement in core specifications.

Very few applications other than GPUs need HBM.

nullsanity 3 hours ago | parent | prev [-]

[dead]

naveen99 14 hours ago | parent | prev | next [-]

But wouldn’t you rather hbm prices come down first ? Memory makers will be fine. There is practically infinite demand. Unless you get china style rationing of compute per person world wide.

The real issue is everyone wanting to upgrade to hbm, ddr5, and nvme5 at the same time.

elorant 2 hours ago | parent | prev | next [-]

The market is already stagnated. Even if OpenAI doesn’t buy what they reserved other players will do so. SK Hynix CEO said there is a 20% gap between supply and demand per year. And that doesn’t account the shock effect that will take place the moment prices normalize and everyone and their dog will go out and start buying inventory to avoid the next crisis. I for one would certainly buy more than I currently need just in case.

quickthrowman 7 hours ago | parent | prev | next [-]

> Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?

I hope they do, they did not have to agree to sell so much RAM to one customer. They’ve been caught colluding and price fixing more than once, I hope they take it in the shorts and new competitors arise or they go bankrupt and new management takes over the existing plants.

Don’t put all your eggs in the one basket is how the old saying goes.

mschuster91 15 hours ago | parent | prev | next [-]

> Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?

We aren't. The remaining memory manufacturers fear getting caught in a "pork cycle" yet again - that is why there's only the three large ones left anyway.

twic an hour ago | parent | next [-]

Surely this can be solved with financial engineering. The memory makers build more capacity, but they finance it with something like floating-rate notes linked to an index of memory prices, or even catastrophe bonds or AT1s. Or more crudely, set up special purpose vehicles to build the extra capacity, and issue convertible bonds from those; if the memory market collapses, investors don't get paid, but they do get a memory factory.

ahartmetz 3 hours ago | parent | prev [-]

If they don't expand capacity much, the only negative consequences I foresee happening for them is that they might lose spending discipline, and that systems will be set up to make do with a little less memory. Apart from that, it's just very high profits followed by more or less regular profits.

XorNot 2 hours ago | parent [-]

They could wind up losing all their business to China though.

China has memory makers who are creeping up through the stages of production maturity, and once they hit then there's no going back.

If the existing makers can't meet supply such that Chinese exports get their foot in the door, they may find they never get ahead again due to volume - that domestic market is huge so they have scale, and the gaming market isn't going to care because they get anything at the moment, which is all you'll need for enterprise to say "are we really afraid of memory in this business?"

shevy-java 4 hours ago | parent | prev | next [-]

Good point. I think both AI companies and hardware makers should pay for the damage they caused to us here.

They act as a de-facto monopoly and milk us. Why is this allowed?

jonas21 3 hours ago | parent | next [-]

It's a business with huge up-front capital expenses and typically very low margins. Supply is scaling up slowly because it's hard, and if you overshoot, you go out of business.

Nobody is "allowing" this. It's a natural property of being both advanced technology and a commodity at the same time.

occamofsandwich 2 hours ago | parent | next [-]

The strange deals on the entire future output are what was allowed. Try to do the same thing with onions and the government understands you are a criminal.

https://en.wikipedia.org/wiki/Onion_Futures_Act

saintfire 10 minutes ago | parent | prev [-]

It has the makings of a natural monopoly, except its compounded by RAM cartels colluding to shut out the last of the competitors.

Recently they had a second price fixing lawsuit thrown out (in the US).

Now with the state of things I'm sure another lawsuit will arrive and be thrown out because the government will do anything to keep the AI bubble rolling and a price fixing suit will be a threat to national security, somehow. Obviously thats speculative and opinion but to be clear, people are allowing it. There are and more so were things that could be done.

estimator7292 3 hours ago | parent | prev | next [-]

Because for the last 60 years we've allowed big business to buy and hollow out our legal and education systems.

kennywinker 4 hours ago | parent | prev [-]

Allowed? We live in a neoliberal world where corporate monopolies / oligopolies aren’t even remotely regulated. If you try to do even the gentlest regulation of companies people scream about communism and totalitarianism. Unless the regulation serves the monopolies by making it harder to enter the market.

It started with raegan, and even parties on the “left” in the west believe in it with very few exceptions.

AnthonyMouse 3 hours ago | parent [-]

> We live in a neoliberal world where corporate monopolies / oligopolies aren’t even remotely regulated. If you try to do even the gentlest regulation of companies people scream about communism and totalitarianism. Unless the regulation serves the monopolies by making it harder to enter the market.

The thing that enables this is pretty obvious. The population is divided into two camps, the first of which holds the heuristic that regulations are "communism and totalitarianism" and this camp is used to prevent e.g. antitrust rules/enforcement. The second camp holds the heuristic that companies need to be aggressively "regulated" and this camp is used to create/sustain rules making it harder to enter the market.

The problem is that ordinary people don't have the resources to dive into the details of any given proposal but the companies do. So what we need is a simple heuristic for ordinary people to distinguish them: Make the majority of "regulations" apply only to companies with more than 20% market share. No one is allowed to dump industrial waste in the river but only dominant companies have bureaucratic reporting requirements etc. Allow private lawsuits against dominant companies for certain offenses but only government-initiated prosecutions against smaller ones, the latter preventing incumbents from miring new challengers in litigation and requiring proof beyond a reasonable doubt.

This even makes logical sense, because most of the rules are attempts to mitigate an uncompetitive market, so applying them to new entrants or markets with >5 competitors is more likely to be deleterious, i.e. drive further consolidation. Whereas if the market is already consolidated then the thicket of rules constrains the incumbents from abusing their dominance in the uncompetitive market while encouraging new entrants who are below the threshold.

moffkalast 15 hours ago | parent | prev | next [-]

Memory makers did get themselves into this situation by selling all wafers for empty promises and alienating everyone but OpenAI tbh. I do hope they end up holding the bag once again, cause after covid and the cartel thing they don't seem to ever learn their lesson on how to have the tiniest amount of integrity.

AnthonyMouse 3 hours ago | parent | next [-]

> Memory makers did get themselves into this situation by selling all wafers for empty promises and alienating everyone but OpenAI tbh.

Wasn't the problem here that OpenAI was negotiating with Samsung and SK Hynix at the same time without the other one knowing about it? People only realized the implications when they announced both deals at once.

gck1 14 hours ago | parent | prev | next [-]

While we're giving away bags, I'd like HDD manufacturers to get some too.

zeristor 7 hours ago | parent | prev [-]

That wouldn’t help if another one goes bankrupt that’ll only make things worse.

moffkalast 7 hours ago | parent [-]

Sounds like they're too big to fail, maybe we should bail them out to reinforce that they will get rewarded for making bad decisions.

atq2119 3 hours ago | parent [-]

Permanent public ownership of (very large stakes in) these companies doesn't seem like such a bad idea anymore, does it? It's what we used to have for most of the 20th century at least in Europe.

Rekindle8090 11 hours ago | parent | prev | next [-]

This will result in demand destruction which will starve the enterprise which will starve the hyperscaler. theres no situation where people not being able to afford hardware for 4 years results in the bubble not popping

hsbauauvhabzb 14 hours ago | parent | prev | next [-]

The people who fucked over consumers are left holding the back that they sold us out over?

Oh no!

VladStanimir 14 hours ago | parent [-]

They won't be, prices are high because they are refusing to build capacity for demand that may evaporated by the time they are done. They are holding back and building only enough so when the bubble pops they will be fine.

DoctorOetker 13 hours ago | parent [-]

So the ML hate is weaponized in the form of memory demand collapse FUD, and the public at large has to pay through their nose for it... thanks party poopers!

VladStanimir 10 hours ago | parent [-]

I don't think its from the ML collapse FUD, its most likely from the multiple time's in the past when they overbuilt and it resulted in a memory oversupply and price collapses. The 1985–1988, 1993–1994, 1998–2002 and the post pandemic oversupply. These were all cases where shortages followed by over corrections caused oversupply, financial losses due to low prices and fewer surviving companies. I think they're taking their time and are cautiously adding more capacity in such a way that prices won't end up collapsing again. Regardless, the result is still that we the consumers have to pay more.

bee_rider 3 hours ago | parent | next [-]

At this point the remaining memory companies are… the ones that didn’t die during an over-supply collapse, right? I guess there’s been a strong evolutionary pressure against giving consumers what we want, haha.

DoctorOetker 7 hours ago | parent | prev [-]

its not like all the RAM is passing the same machine, they can gradually increase machines and observe the change in demand, and smoothly match it.

AnthonyMouse 3 hours ago | parent [-]

If they gradually increase production capacity then prices stay high for 10+ years (or for as long as it takes for demand to crash) because a gradual increase in production takes that long for them to add enough capacity for current demand.

If they add enough capacity to meet current demand quickly then if demand crashes they still have billions of dollars in loans used to build capacity for demand that no longer exists and then they go bankrupt.

The biggest problem is predicting future demand, because it often declines quickly rather than gradually.

DoctorOetker 24 minutes ago | parent [-]

do we have evidence of RAM manufacturers going bankrupt? do we have evidence that the increased capacities after the mentioned past shortages went unused or were operated at a loss?

kubb 15 hours ago | parent | prev [-]

I would expect that OpenAI gets as much money as they ask for for the next 10 years.

There’s virtually infinite capital: if needed, more can be reallocated from the federal government (funded with debt), from public companies (funded with people’s retirement funds), from people’s pockets via wealth redistribution upwards, from offshore investment.

They will be allowed to strangle any part of the supply chain they want.

torginus 14 hours ago | parent | next [-]

China already has a well developed DRAM industry, as DRAM is somewhat easier than logic, and can tolerate a much higher defect rate. The industry will figure this out.

Another point is I often see the money argument - like country X has more money, so they can afford to do more and better R&D, make more stuff.

This stuff comes out of factories, that need to be built, the machinery procured, engineers trained and hired.

jimnotgym 9 hours ago | parent [-]

If China capitalises on the big three focusing on data centre team, the big three might have a very hard time post bubble

rzerowan 3 hours ago | parent [-]

I think the article has a giant blind spot as far as China is concerned , considering they have already a mature enough memory ecosystem via YMTC that Apple was considering sourcing from them. As well as continued expansion in the DRAM and HBM Fabs [1]. It feels like the memory cartel once again trying to incentivise their various govt to cough up some more tax breaks/funding to cushion the AI buildout bet that they made and the bubble seeming about to pop. In any case if they leave the consumer market underserved it should be no surprise if before that 2030 prediction we are all on cheaper YMTC memory modules.

[1]https://www.tomshardware.com/tech-industry/semiconductors/ym...

platevoltage 3 hours ago | parent | prev | next [-]

I'm guessing they become pets.com within the year. At least I hope.

IshKebab 15 hours ago | parent | prev | next [-]

Maybe if they had no competitors...

SlinkyOnStairs 14 hours ago | parent | prev | next [-]

I think you're massively overestimating how much money is really accessible here. The parent comment's right that all of the easily available VC & private equity investment is basically used up. OpenAI was struggling to sell $600M of private equity, the big multi-billion dollar investment packages had lots of conditions and non-cash in it.

> more can be reallocated from the federal government (funded with debt)

While this is the most reliable funding, it's still not very accessible. OpenAI is a money pit, and their demands are growing quickly. The US government has started a bunch of very expensive spending. If OpenAI were to require yearly bundles of it's recent "$120B" deal, that's 6% of the US' discretionary budget. 12.5% of the non-military discretionary budget. (And the military is going to ask for a lot more money this year) Even the idea of just issuing more debt is dubious because they're going to want to do that to pay for the wars that are rapidly spiralling out of control.

None of this is saying that the US government can't or wouldn't pay for it, but it's non trivial and it's unclear how much Altman can threaten the US government "give me a trillion dollars or the economy explodes" without consequences.

Further deficit-spending isn't without it's risks for the US government either. Interests rates are already creeping up, and a careless explosion of deficit may well trigger a debt crisis.

> from public companies (funded with people’s retirement funds)

This would be at great cost. OpenAI would need to open up about it's financial performance to go public itself. With it's CFO being put on what is effectively Administrative Leave for pushing against going public, we can assume the financials are so catastrophic an IPO might bomb and take the company down with it. Nobody's going to be investing privately in a company that has no public takers.

Getting money through other companies is also running into limits. Big Tech has deep pockets but they've already started slowing down, switching to debt to finance AI investment, and similarly are increasingly pressured by their own shareholders to show results.

> from people’s pockets via wealth redistribution upwards

The practical mechanism of this is "AI companies raise their prices". That might also just crash the bubble if demand evaporates. For all the hype, the productivity benefit hasn't really shown up in economy-wide aggregates. The moment AI becomes "expensive", all the casual users will drop it. And the non-casual users are likely to follow. The idea of "AI tokens" as a job perk is cute, but exceedingly few are going to accept lower salary in order to use AI at their job.

There's simply not much money to take out of people's pockets these days, with how high cost of living has gotten.

> from offshore investment.

This is a pretty good source of money. The wealthy Arabian oil states have very deep slush funds, extensively investing in AI to get ties to US businesses and in the hope of diversifying their resource economies.

...

...

"Was". Was a good source of money.

kubb 13 hours ago | parent [-]

I'm genuinely curious to find out how many billions they get every year from now.

saidnooneever 14 hours ago | parent | prev [-]

love that theres virtually infinite capital there. meanwhile in the rest of the world there is virtually no food.

convolvatron 3 hours ago | parent [-]

are you kidding? if spent all that money on food you guys would just use it to bullshit all day and make funny pictures, while if we spend it on AI..