Remix.run Logo
▲ gpt5 6 hours ago

Data centers are becoming higher priority targets in wars recently, and with humans delegating more to AI, I suspect their priority will increase even further.

Makes me wonder how much compute is already being built in bunkers under mountains.

▲buildfocus 5 hours ago | parent | next [-]

Scaleway built one of their Paris AZs in a nuclear fallout shelter: https://www.scaleway.com/en/blog/meet-fr-par-3/

▲MiroslavPokorny 6 hours ago | parent | prev | next [-]

Datacenters are full of valuable computers and related equipment, but building the equivalent space under a mountain will cost a crazy amount.

Not worth paying 100x for the bunker to save 1x worth of computer stuff.

▲walrus01 6 hours ago | parent | next [-]

Building any appreciable amount of compute capacity (on the scale of, let's say, a single large datacenter in northern VA) inside a mountain also becomes a huge heat rejection problem, since with the exception of the 0.00000000001% that leaves the facility as photons down a SM fiber cable, 100% of the energy consumed is turned into heat.

So you build your stuff inside a mountain and run a bunch of big ass liquid cooling loops to heat exchangers in the atmosphere outside the mountain. The requirement of the cooling system being out in the open air means it can also be hit by air strikes, an adversary doesn't need to penetrate the mountain, just take out its cooling.

You'd have to build a fully underground cooling system that goes to a nearby river (or huge lake) and use the cold water from that, which would get costly quick.

▲jltsiren 5 hours ago | parent | next [-]

> You'd have to build a fully underground cooling system that goes to a nearby river (or huge lake) and use the cold water from that, which would get costly quick.

Helsinki gets its water from the 120 km Päijänne Water Tunnel, which cost €200-250 million (adjusted for inflation) to build. Peak flow is 10 m^3/s by gravity alone and up to 20 m^3/s with pumps, which should be enough to cool a 1 GW data center. You could increase the cost by 2x (because real wages have gone up), by another 2x (because middlemen have destroyed productivity), and by yet another 2x (because everything is more expensive in America), and it would still be pretty reasonable.

▲Brybry 4 hours ago | parent | prev | next [-]

Iron Mountain does it but they used an existing limestone mine with a large water reservoir [1][2]

Edit: I guess the scale isn't comparable to a, for example, Google data center though.

[1] https://www.ironmountain.com/data-centers/colocation/our-dat...

[2] https://betterbuildingssolutioncenter.energy.gov/showcase-pr...

▲killerstorm 2 hours ago | parent | prev | next [-]

They already run liquid cooling loops - compute density is way beyond what could circulate through air. Adding more pipes is probably trivial on the scale of datacenter expenses

▲walrus01 2 hours ago | parent [-]

Liquid cooling loops don't solve the problem of needing to reject the heat to the atmosphere outside of the building. It just makes it more efficient to transport the heat. At the grossest possible definition, a datacenter of N megawatts on Y acres of land is going to emit the same amount of heat whether it's air cooled equipment or liquid is used to take the heat to the outdoor heat exchanging equipment. On the scale of any very large datacenter the heat exchanging equipment is visible from aerial photos (and from space), take a look at google maps for some of the largest hyperscalers for examples.

▲killerstorm an hour ago | parent [-]

I assume that "liquid cooling loop" has an endpoint outside of the building which hosts hardware. In this case it's exactly same problem - just make pipes longer.

Do you need many acres of land to host a heat exchanger?

▲efnx 5 hours ago | parent | prev | next [-]

So they want to build it in space. Trade one heat problem for another, but put it completely out of range (for most attackers).

▲martinjc 5 hours ago | parent | prev | next [-]

Pionen in sweden is a datacenter in a bunker. The Pirate Bay used to be hosted there.

▲walrus01 5 hours ago | parent [-]

A few racks or even a couple of dozen racks could be called a datacenter, but nothing on any real scale. Enough to host important and niche things? Sure. I was referring more to like the size of what you might see in Quincy, WA.

▲jeltz 4 hours ago | parent | next [-]

Pionen is small but it is quite a bit bigger than "a couple of dozen racks". More like 100-200 if I recall correctly.

▲jacquesm an hour ago | parent [-]

By 1990's standards that was small. Today it is tiny.

▲martinjc 24 minutes ago | parent [-]

In the US the most common deployed size is below 5MW?

▲jacquesm 7 minutes ago | parent [-]

Yes, but that's not where the volume is. A 'small' data center, even here in Europe has 100's of racks. There are of course many tiny ones but collectively they don't even add up to one single large Google or AWS DC. Those things will run in excess of 1 or even 2 GW.

▲martinjc 4 hours ago | parent | prev [-]

According to their site, Pionen has 1.5MW of cooling capacity https://swedendedicated.com/datacenter/

▲testaccount28 6 hours ago | parent | prev [-]

neat. why can't you put the heat in the mountain?

▲walrus01 5 hours ago | parent [-]

For broadly the same reason that many years of heat from braking has heat-saturated the london underground.

https://www.google.com/search?client=firefox-b-d&q=london+un...

You could probably drill sideways into rocks and pump heat into the mountain for a certain period of time, but on a multi-month or year time scale, it would catch up with you after you had saturated it.

▲jazzyjackson 6 hours ago | parent | prev | next [-]

The bunkers are already out there e.g. limestone mines in US, Italy,

https://www.datacenterknowledge.com/build-design/safer-under...

▲walrus01 6 hours ago | parent [-]

Lots of people have built commercial, non-military, boutique scale stuff in hardened facilities, nobody I'm aware of has done it at scale. There's certainly money to be made from selling expensive "we put datacenter stuff deep down inside a mine" maybe to some weird enterprise and government customers, so the revenue per rack (or per theoretical 208V 30A circuit?) is probably a lot more than commodity colo. But I would very much like to see the aerial photos, if anyone can link them, to a claimed large scale underground or hardened datacenter.

It's a severe engineering challenge to hide massive heat exchangers on the scale of the datacenters built in the industrial area near Dulles (coordinate example: 38.9817915,-77.417775). Find those on a map and observe the scale of their cooling and generator systems and then try to find anything comparable that's been built underground.

▲boelboel 4 hours ago | parent | next [-]

There's some 200MW capacity (only 80MW is currently used or being built) data center in a mine near a fjord in Norway. The limiting factor for its capacity seems to be the capacity of the electricity net nearby (which you can just hit and the DC is useless). With how NIMBY Norwegians are there's no way it would ever get to a GW or more even if it might be both technically and economically feasible. That's just one project and can't be replicated anywhere else. It's just a massive advantage that above ground datacenters are relatively 'figured out' and similar everywhere.

edit: 61.9340107, 5.5084207 are the coordinates

▲walrus01 23 minutes ago | parent [-]

Do you know if that 200MW figure is total power budget for the facility (including electricity used for cooling systems and other facilities-support systems), or up to 200MW of electronics loads generating heat? The latter would be something like 300 cabinets with 5-6kW heat each.

▲boelboel 10 minutes ago | parent [-]

Apparently it's for the whole facility, they have a power usage effectiveness of around 1.1 so 175-180MW of electronics.

https://new.abb.com/news/detail/5924/abb-solutions-power-eur...

▲jazzyjackson 6 hours ago | parent | prev | next [-]

Agreed it’s small potatoes, and as a sibling mentions you will still have infrastructure exposed that poses a target to disable the data center, unless you were to drill geothermal and find an underground river to cool it.

Just wanted to make the point you don’t have to pay to excavate the cave :)

▲misiek08 5 hours ago | parent | prev | next [-]

Germans hid plane engine factories underground with old (not so efficient) diesel generators running 24/7. 80 years ago, so the tech was leaner. Also they didn’t have issues with ventilation and even today if you don’t know where - you won’t see anything driving around.

▲IshKebab 5 hours ago | parent | prev [-]

That's probably just because there's not much demand for it, not because it's difficult.

There are mines with plenty of space in them, and heat removal can be done with water cooling. But who is going to pay 50% more for that when it isn't needed 99.99% of the time?

▲walrus01 5 hours ago | parent [-]

Or when the same thing can be accomplished by putting nothing in a mine, but an approach as simple as having geographical diversity by paying for some 'normal' colo in Hillsboro, and more normal colo in northern VA on the other side of the continent, and duplicating your stuff between two facilities. Weird underground datacenters aren't exactly known for having a large number of competing on-net carriers.

▲jeltz 4 hours ago | parent [-]

> Weird underground datacenters aren't exactly known for having a large number of competing on-net carriers.

Huh? The ones I know of have plenty of connectivity. The real issue is your first, that people prefer to just put their stuff in two countries over paying premium for having their servers under ground.

▲walrus01 4 hours ago | parent [-]

Your definition of connectivity may not be the same as mine, I'm referring to the number of facilities based carriers and those who ride on top of them you can get at major sites in Hillsboro. Facilities with peeringdb pages that have tons of on net carriers. Is there even an underground bunker colo that's big enough to rate its own peeringdb facility page?

▲roenxi 6 hours ago | parent | prev | next [-]

It might be possible to do on the cheap; underground mines get decommissioned from time to time and could be repurposed as a data centre.

Although I personally would expect some sort of decentralisation option to be more effective. It's difficult to even theorise because we don't know exactly what sort of damage these strikes are doing and what countermeasures are already in place. If this strike has caught Yandex by surprise after almost 5 years of hot war then they really should hire a dude to check up on current events every so often. They must have some sort of contingency planning around a Ukrainian strike.

▲microtonal 6 hours ago | parent [-]

It's sad that we have moved away so far from the decentralized web from the 90s and the P2P boom of the 00s. Just like solar panels on every house as a source of energy are much harder to take out than just a few power plants, local P2P storage, messaging, source management, etc. are much harder to take out.

But yeah, the economic incentives are creating unicorns and centralizing compute/capital/etc.

▲jeltz 4 hours ago | parent [-]

The war in Ukraine had showed us that distributed power generation, like solar, and nuclear are most resilient. Nuclear because people do not dare striking the power plants.

▲ 6 hours ago | parent | prev [-]
[deleted]
▲kolinko 2 hours ago | parent | prev | next [-]

With AI there is little data in datacenters - the only thing you lose is % compute.

▲nixass 11 minutes ago | parent [-]

Datacenters are not only "data"

▲PowerElectronix 2 hours ago | parent | prev | next [-]

In bunkers, IDK. In places far away from conflict or conflictive nations, a lot.

▲conmod278 2 hours ago | parent | prev | next [-]

You are also forgetting the undersea cables.

▲paaloeye 6 hours ago | parent | prev | next [-]

They will probably start building DCs in Siberia. Cheap cooling!

▲microtonal 5 hours ago | parent [-]

That's probably not going to help much. Ukraine has been able to hit targets very far inland with drones. They are also rapidly making advances in ballistic missile development. FP-7 has a range of 200-300km, their new FP-9 will have a range of up to 855km and an accuracy of 20m. I know that Russia is (extremely) large, but with development going at this pace...

▲youngtaff 5 hours ago | parent | next [-]

Ukraine are hitting targets 2,500km away

https://united24media.com/war-in-ukraine/ukraine-hits-russia...

▲paaloeye 5 hours ago | parent | prev [-]

dunno, Moscow to Novosibirsk is around 1,800 miles straight-line distance. The war would be probably over by the time Ukraine would have developed those capabilities.

▲boelboel 5 hours ago | parent [-]

Novosibirsk only has cheap cooling in the winter, in the summer months it is hot. Better to go somewhere like Sabetta where they have their LNG harbour.

▲toomuchtodo 6 hours ago | parent | prev | next [-]

Power and data has to come from somewhere for those bunkers. Every fiber hut and power substation becomes a target. Can they all be protected? No.

▲jeltz 4 hours ago | parent [-]

Fibers? Yes, they are easy to protect. The hard part is power generation.

▲toomuchtodo 4 hours ago | parent [-]

Buried fiber, hard to attack. Fiber repeater shelters, easy to attack. Their locations are public information, and any non public are trivial to find, in my experience as a cybersecurity professional who seeks out infra vulnerabilities for reporting for fun. Public records, satellite imagery, all OSINT one can do from a workstation.

You could potentially bury them in the ground, assuming you don’t use a generator for backup power. To my knowledge, this is not done today except in urban areas where an underground vault is used for power and comms.

https://www.decommsystems.com/resources/what-goes-into-an-il...

▲oytis 6 hours ago | parent | prev | next [-]

I don't think Yandex DCs are used directly for military purposes much. It's more about harming Russian economy in general, including its highly prided digital infrastructure

▲ponector 6 hours ago | parent | next [-]

Military personnel use the same Yandex services as regular people: delivery, maps, taxi, etc.

Also Yandex is a large cloud provider, they host a lot of government-connected companies.

▲andy_ppp 6 hours ago | parent | prev | next [-]

Also Russia has been targeting Ukrainian digital infrastructure so they make these things fair game IMO.

▲youngtaff 5 hours ago | parent [-]

Russia is hitting everything in Ukraine… DC, power plants, rail, road housing

Ukraine has a very targeted response but notably tries to minimise civilian casualties

▲fartfeatures 6 hours ago | parent | prev [-]

I don't have specific information about Yandex but don't forget how heavily involved Amazon were with things like: https://www.nytimes.com/2022/12/19/world/europe/ukraine-cybe... it wouldn't be a huge stretch to suggest that a lot of cloud hosting providers are neck deep in national security.

▲crabbone 5 hours ago | parent | prev | next [-]

This is a... mischaracterization.

The reason why Ukraine, Russia and Iran are targeting data-centers is because they are easy targets with a lot of expensive equipment, but they don't really impact the military operation. So, they aren't high priority targets. Bomber airplanes, air defenses, launchers for ballistic missiles, command centers, factories manufacturing strategic military equipment, telecommunications, energy infrastructure would be the examples of high priority targets. The data-centers are more of a "we have excess drones, why not use them?"

I find it hard to believe that the military will intentionally use civilian data-centers for its own purposes. At least, not for the core of its operations (for auxiliary functionality: maybe). Military data-centers are likely to be in protected locations, perhaps using different h/w that addresses the cooling problem.

Also, it's a bit myopic to think that AI is having such a universal and large impact on society. Jobs that provide essential goods and services are, by and large, unaffected by AI. Farmers, construction workers, cooks etc. don't really use any of the AI stuff on the day to day basis, if at all.

▲scotty79 3 hours ago | parent [-]

Those data centers participate hugely in generating of internet misinformation and propaganda that shapes deterioration of support in European populations. Those are valid military targets.

▲bob1029 5 hours ago | parent | prev [-]

Let's not forget about the orbital data center idea.

Perhaps an economic justification can be found for placing something so far up the gravity well that its adversaries can no longer physically approach it. Distributing the resource across many satellites would further confound any attack.

Increasing the cost of effect on a single target from $1,000 to $1,000,000,000 makes a difference. Especially when that 2nd option is likely to trigger additional multilateral strategic responses. If the only way to delete your adversaries data centers is with things like nuclear weapons, lasers and multi-stage rocketry, your adversary may have already won the game.

▲Gigachad 5 hours ago | parent | next [-]

Until someone blows up a satellite and creates enough shrapnel that every other satellite is destroyed.

▲ben_w 5 hours ago | parent | prev | next [-]

Ground to orbit antisat lasers are already plausibly something various militaries have.

Welding lasers are pretty cheap, the two hard parts are a big enough apperture to focus 500 km away, and adaptive optics, both of which have already been demonstrated decades ago in astronomy.

I'm not saying Iran and North Korea already have such capabilities, but I am saying I think neither would have trouble building it six months ago if they'd wanted to.

▲microtonal 5 hours ago | parent | prev [-]

Except that doing this at scale is currently still a pipe dream. Space is a near-vacuum, so it's hard to get rid of the heat.

The active thermal control system (ATCS) on the ISS uses an ammonia loop and radiators measuring 13.6 x 3.12 meters, or about 42.5 square meters. It can dissipate 16 kW, which is enough for 16 H200s, or one-quarter of a rack on Earth. For 200 kW, you would need a system 12.5 times larger, about 531 square meters—2.6 times larger than the solar array. The satellite would be enormous, larger than the ISS, and all of that for only three racks.

https://codedtrip.com/en/blog/data-centers-in-space-disastro...

▲Marha01 5 hours ago | parent | next [-]

> For 200 kW, you would need a system 12.5 times larger, about 531 square meters—2.6 times larger than the solar array. The satellite would be enormous, larger than the ISS, and all of that for only three racks.

Starlink gen 3 satellite is designed to dissipate 250 kW. You are talking about an already solved issue, with real hardware already being built today.

▲bob1029 5 hours ago | parent | prev | next [-]

Are you asserting that we must use the exact same heat rejection architecture the ISS uses for data centers?

https://en.wikipedia.org/wiki/Liquid_droplet_radiator

▲walrus01 5 hours ago | parent | next [-]

The problem of the $ per kg to launch huge radiators and "orbital datacenter" type stuff of satellites into space doesn't seem to be resolving itself any time soon. Even if we hand wave away and say that 5 years from now the starship will be such a spectacular success than it costs only $200 per kg to send stuff into space (drastically lower than what it costs now), it would still be vastly less expensive to build a terrestrial solar datacenter in the middle of the southern Libyan desert than it would be to do the same amount of compute in space.

▲Marha01 5 hours ago | parent | next [-]

> it would still be vastly less expensive to build a terrestrial solar datacenter in the middle of the southern Libyan desert than it would be to do the same amount of compute in space.

Perhaps, but the parent comment posited that some people would pay the premium for orbital data center due to much better security. At $200 per kg to orbit, it could make sense.

▲fragmede an hour ago | parent | prev [-]

With the caveat that Libya, being located upon Earth, is affected by solar power not working at night or when it's cloudy, and is also subject to being in Libya, which is a country with bureaucrats that are going to subject anything built in the country to their laws.

▲evgen 5 hours ago | parent | prev [-]

So the "some day in the future" orbiting data centers proposed by a known fraudster will use an experimental-at-best radiator technology that has never been tested in flight yet? That tracks.

▲Marha01 5 hours ago | parent [-]

Orbital data center constellations are also studied by Google, the idea is not just from Elon.

▲Marha01 5 hours ago | parent | prev [-]

> Space is a near-vacuum, so it's hard to get rid of the heat.

It's not that hard, Starlink constellation already dissipates around 100 MW of heat, split over many satellites. Any orbital datacenter would use similarly distributed satellites.