Remix.run Logo
Remix clone Hacker News
new | show | ask | jobsGithub
▲ elbasti 4 days ago

I work in the datacenter space. The power consumption of a data center is the "canonical" way to describe their size.

Almost every component in a datacenter is upgradeable—in fact, the compute itself only has a lifespan of ~5 years—but the power requirements are basically locked-in. A 200MW data center will always be a 200MW data center, even though the flops it computes will increase.

The fact that we use this unit really nails the fact that AI is basically refining energy.

▲aurareturn 4 days ago | parent | next [-]

  A 200MW data center will always be a 200MW data center, even though the flops it computes will increase.
This here underscores how important TSMC's upcoming N2 node is. It only increases chip density by ~1.15x (very small relative to previous nodes advancements) but it uses 36% less energy at the same speed as N3 or 18% faster than N3 at the same energy. It's coming at the right time for AI chips used by consumers and energy starved data centers.

N2 is shaping up to be TSMC's most important node since N7.

▲alberth 4 days ago | parent [-]

> N2 is shaping up to be TSMC's most important node since N7

Is it?

N2, from an energy & perf improvement seems on par with any generation node update.

          N2:N3   N3:N5  N5:N7
  Power   ~30%    ~30%    ~30%
  Perf    ~15%    ~15%    ~15%
https://www.tomshardware.com/news/tsmc-reveals-2nm-fabricati...
▲aurareturn 4 days ago | parent [-]

Yes. It has more tape outs at this stage of development than both N5 or N3. It’s wildly popular for chip designers it seems.

▲alberth 3 days ago | parent [-]

I thought Apple gets exclusive access to the latest node for the first 1-2 years. Is that not the case?

▲aurareturn 3 days ago | parent [-]

No. That's not the case. Maybe for a few months only.

▲alberth 3 days ago | parent [-]

Correct me if I'm wrong but didn't TSMC launch N3 in 2022, and still only Apple uses this latest/smallest node.

Both AMD and NVIDIA are using N4.

▲aurareturn 2 days ago | parent [-]

Apple, Mediatek, Qualcomm, Intel

▲pseudosavant 4 days ago | parent | prev | next [-]

I love that term "refining energy". We need to plan for massive growth in electricity production to have the supply to refine.

▲tmalsburg2 4 days ago | parent [-]

Sounds smart but it’s abusing the semantics of “refine” and is therefore ultimately vacuous.

▲pseudosavant 3 days ago | parent [-]

I think it is really just the difference between chemically refining something and electrically refining something.

Raw AC comes in, then gets stepped down, filtered, converted into DC rails, gated, timed, and pulsed. That’s already an industrial refinement process. The "crude" incoming power is shaped into the precise, stable forms that CPUs, GPUs, RAM, storage, and networking can actually use.

Then those stable voltages get flipped billions of times per second into ordered states, which become instructions, models, inferences, and other high-value "product."

It sure seems like series of processes for refining something.

▲jacquesm 4 days ago | parent | prev | next [-]

It is the opposite of refining energy. Electrical energy is steak, what leaves the datacenter is heat, the lowest form of energy that we might still have a use for in that concentration (but most likely we are just dumping it in the atmosphere).

Refining is taking a lower quality energy source and turning it into a higher quality one.

What you could argue is that it adds value to bits. But the bits themselves, their state is what matters, not the energy that transports them.

▲elbasti 4 days ago | parent [-]

I think you're pushing the metaphor a bit far, but the parallel was to something like ore.

A power plant "mines" electron, which the data center then refines into words. or whatever. The point is that energy is the raw material that flows into data centers.

▲fuzzfactor 4 days ago | parent [-]

Maybe more like converting energy to data, as a more specific type of refinement.

▲phkahler 4 days ago | parent [-]

Using energy to decrease the entropy of data. Or to organize and structure data.

▲LaGrange 3 days ago | parent | next [-]

This is OpenAI, they are not decreasing the entropy. This is refining coal into waste heat and CO2.

▲fuzzfactor 4 days ago | parent | prev [-]

I like that. Take random wild electrons and put them neatly into rows & columns where they can sit a spell.

▲reubenmorais 4 days ago | parent | prev | next [-]

All life is basically refining energy - standing up to entropy and temporarily winning the fight.

▲HPsquared 4 days ago | parent | next [-]

It's all about putting the entropy somewhere else and keeping your own little area organised.

▲xnickb 4 days ago | parent [-]

People of the earth, remember: unnecessary arm and leg movements increase the entropy! Fear of the heat death of the universe! Lie down when possible!

▲antihipocrat 4 days ago | parent | prev | next [-]

Yes, in a very local context it appears so, but net entropy across the system from life's activities is increased

▲ithkuil 4 days ago | parent | prev [-]

"the purpose of life is to hydrogenate carbon dioxide"

-- Michael Russel

▲casey2 3 days ago | parent | prev | next [-]

Where do the cards go after 5 years? I don't see a large surplus of mid sized cloud providers coming to buy them (cause AI isn't profitable), Maybe other countries (possibly illegally)? Flood the consumer market with cards they can't use? TSMCs' more than doubled packaging and they are planning on doubling again

▲protocolture 4 days ago | parent | prev | next [-]

This.

A local to me ~40W datacenter used to be in really high demand, and despite having excess rack space, had no excess power. It was crazy.

▲nixass 4 days ago | parent [-]

40W - is that ant datacenter? :)

▲protocolture 4 days ago | parent [-]

Yeah, it was the companies pilot site, and everything about it is tiny.

But it very quickly became the best place in town for carrier interconnection. So every carrier wanted in.

Even when bigger local DC's went in, a lot of what they were doing was just landing virtual cross connects to the tiny one, because thats where everyone was.

▲inemesitaffia 4 days ago | parent [-]

You lost a M or K next to your W.

I still have an Edison bulb that consumes more power.

▲protocolture 4 days ago | parent [-]

Yep I see that haha.

▲pabs3 4 days ago | parent | prev | next [-]

> the power requirements are basically locked-in

Why is that? To do with the incoming power feed or something else?

▲brendoelfrendo 4 days ago | parent | next [-]

Basically, yes. When you stand up something that big, you need to work with the local utilities to ensure they have the capacity for what you're doing. While you can ask for more power later on, if the utilities can't supply it or the grid can't transport it, you're SOL.

▲pabs3 3 days ago | parent [-]

You could in theory supplement it with rooftop solar and batteries, especially if you can get customers who can curtail their energy use easily. Datacentres have a lot of roof space, they could at least reduce their daytime energy costs a bit. I wonder why you don't see many doing solar, do the economics not work out yet?

▲brendoelfrendo 3 days ago | parent | next [-]

I'd have to do the math, but I doubt that makes sense given the amount of power these things are drawing. I've heard of DCs having on-site power generation, but it's usually in the form of diesel generators used for supplemental or emergency power. In one weird case, I heard about a DC that used on-site diesel as primary power and used the grid as backup.

▲XorNot 3 days ago | parent | prev [-]

Compared to their volume they absolutely do not: you get about ~1kW / m^2 of solar. Some quick googling suggests a typical DC workload would be about 50 kW / m^2, rising too 100 for AI workloads.

▲jl6 4 days ago | parent | prev | next [-]

Cooling too. A datacenter that takes 200MW in has to dissipate 200MW of heat to somewhere.

▲djtriptych 4 days ago | parent | prev [-]

guessing massive capital outlays and maybe irreversible site selection/preparation concerns.

▲kulahan 4 days ago | parent | prev | next [-]

That's pretty interesting. Is it just because the power channels are the most fundamental aspect of the building? I'm sorta surprised you can't rip out old cables and drop in new ones, or something to that effect, but I also know NOTHING about electricity.

▲libraryofbabel 4 days ago | parent [-]

Not an expert, but it’s probably related to cooling. Every joule of that electricity that goes in must also leave the datacenter as heat. And the whole design of a datacenter is centered around cooling requirements.

▲vrighter 3 days ago | parent [-]

Exactly. To add to that, I'd like to point out that when this person says every joule, he is not exaggerating (only a teeny tiny bit). The actual computation itself barely uses any energy at all.

▲pjc50 3 days ago | parent | prev [-]

Refining it into what? Stock prices?