Remix.run Logo
bayindirh an hour ago

> or even resources like water or power

I wonder, do you know how much energy or water is required to run a datacenter with direct liquid cooled systems?

Not the SOTA AI data centers which are popping up like mushrooms, a general ballpark is enough.

This is not a loaded question. Given your knowledge, you must be knowing much more than majority of HN.

jpgvm an hour ago | parent [-]

I actually happen to know a lot about this! I have been in and around datacenters for ~16yrs and I work on direct liquid cooled hardware at $DAY_JOB.

Back when I was running my own startup I moved to Mascot, Sydney to be near Equinix SY3 that was being built which we had bought a decent chunk of committed capacity in. Mascot was originally an industrial area but was gentrified a lot over the last 20ish years but the datacenters, meat packers etc are all still there. It really adds to the charm of the place in a lot of ways, warehouses/factories that have been converted to cafes and breweries. Which is to say I don't think living near a DC is bad at all, they aren't very loud and they just look like big warehouses with lots of lights, cameras and no windows. Oh and security dudes.

Anyways. Yes. The hardware I am working at $DAY_JOB is all direct liquid cooled. The coolant that actually cools the hardware is closed loop, it's not expended. Which means you don't actually use any of it and it's considered good for the entire life of the hardware.

I normally don't directly talk about $DAY_JOB things because there are pretty strict rules so I can only use publically referenced sources but you can read a little bit about the coolant we use in this article [1].

The facility loop though is usually what people are worried about. A lot of datacentres that were built in low humidity environments use evaporative cooling which does indeed use a ton of water. However most data centers in more humid environments don't and either use loops that recycle the water but dump the heat (i.e rivers/lakes) or use closed loop systems with radiators etc. It's worth mentioning though that evaporative cooling is extremely power efficient vs most closed loop solutions so there is a tradeoff here. I don't think Google could achieve their fleetwide PUE (Power Usage Efficiency, basically measurement of how much power goes to things in a DC that isn't a computer) numbers without relying on it heavily.

Given the difficulties getting DCs approved in much of the world right now and the backlog on generation capacity (i.e power) almost all modern DCs are being built to use as little water as possible.

To give you an idea I think evaporative cooling of the type you would find in a state of the art Google DC with a PUE of ~1.1ish might use 1-1.1L of water for every kWh of actual equipment load (this is a back of the napkin estimate, don't hold me to it).

Conversely true closed loop will use approximately zero.

So yeah, evaporative is a big waster of water especially at GW scale.

1: https://www.tomshardware.com/tech-industry/data-centers/nvid...

bayindirh an hour ago | parent [-]

Thanks a ton, for the information. It's greatly appreciated.

I'm personally not interested in the details that you can't provide, that was not something I was aiming for, but putting a ballpark number for ~1.1L/kWh is very eye opening (I know it's not some accurate figure, but yeah). Considering a SOTA GPU uses somewhere from 700W to 2.5kW depending on the manufacturer and performance level...

Now I'm wondering how the dumped heat affect these water bodies (i.e. rivers, lakes, etc.). Their relative heat capacity w.r.t. this amount dumped heat can't be infinite so they can handle this forever.

Hope to meet you someday. Have a nice day! :)