Remix.run Logo
AWS stops billing Middle East cloud customers as repairs to war damage drag on(arstechnica.com)
113 points by johnbarron 5 hours ago | 44 comments
time0ut 4 hours ago | parent | next [-]

Data centers are such great targets in modern warfare. A few cheap drones can inflict billions in damage with low direct casualties (if the attacker even cares). I have heard AWS in particular is secretive about the exact location of their data centers, but no doubt every major country knows exactly where they are.

PunchyHamster 6 minutes ago | parent | next [-]

Hell, you just need to hit a transformer or two to take it entirely down for days if not weeks

cheney_2004 3 hours ago | parent | prev | next [-]

Which is why peace and diplomacy is so important. The last thing we need is to be war hardening everything, which is likely impossible in this day and age.

croisillon 2 hours ago | parent [-]

your handle tho :D

lostlogin an hour ago | parent [-]

Cheney is just salty that targets weren't hardened before he shot his friend in the face.

jmalicki 3 hours ago | parent | prev | next [-]

Just look on satellites for giant buildings with no cars or semi trailers parked in the parking lots.

I wonder if data centers will have to start doubling as automobile junkyards to conceal themselves.

time0ut an hour ago | parent | next [-]

Some data centers are more valuable as targets than others. For example, those comprising us-gov-east-1 and us-gov-west-1 or, god forbid, us-east-1. I don’t expect it is a difficult task to find them and other critical infrastructure for a state, but probably more involved than popping open google maps.

jmalicki an hour ago | parent | next [-]

The US government's security priority should be moving critical us-east-1 services into Cheyenne Mountain.

jdxcode 33 minutes ago | parent | prev [-]

I've always wondered what "us-east-1" is, presumably it's more than just 1 building

throwaway894345 2 hours ago | parent | prev [-]

Some Paris data centers are disguised as apartment buildings with the classic Hausmannian facade, and then you open up Google maps and see a ton of AC units stacked on the roof. These aren’t likely major cloud data centers mind you, and the motivation for concealing them has more to do with the city’s aesthetic codes than military defense.

breppp an hour ago | parent [-]

or less aesthetic versions https://en.wikipedia.org/wiki/33_Thomas_Street#/media/File:A...

robotnikman an hour ago | parent | prev | next [-]

I'm surprised they did not build them in a bunker or other hardened location. The region is not exactly known for its political stability.

fvdessen 2 hours ago | parent | prev | next [-]

That's why the SWIFT backup data centers in Belgium are camouflaged as posh villas (or so i've heard)

amelius an hour ago | parent | prev | next [-]

Can't you use internet geolocation from a distributed botnet, and triangulation based on timing?

dantillberg an hour ago | parent [-]

Yes, and this could perhaps determine location within a few miles.

kdheiwns 4 hours ago | parent | prev | next [-]

Big tech's love for cheap labor is a great mechanism for finding where all their most valuable assets are and mapping out any and all vulnerabilities. I imagine state actors are applying to any and all low paying jobs that have seemingly juicy job requirements and feeling out details during interviews. Even better if you offer to accept a salary far below standard rates and actually get the job.

While probably not a state agent, I've personally done online interviews with some people that were clearly lying about everything and trying to feel out details about the company. People claiming to live in our country and being citizens but having little ability with the language, saying they would love to come to our city but it's a bit far, saying they graduated from a major university but being unable to describe anything about the town (with their resume mentioning graduating from a different university, and their LinkedIn a different university from either), random people moving around and arguing in the background, all their work was with random crypto businesses that shut down within months. I had to stop my coworkers from saying too much. I had to convince them why hiring that person for remote work and giving them access to our servers was a bad idea. There are without a doubt companies giving similar people physical access to their hardware. And there are undoubtedly people who practice interviewing to better deceive companies.

cute_boi 4 hours ago | parent | prev | next [-]

I wonder if you can uncover where the data center is just by using ping command.

yabones 4 hours ago | parent | next [-]

It could give you a rough idea, but it's far from precise. The delay added by a single router could throw you off by several KM.

It's much more effective to just go through satellite imagery and land title records.

https://www.wolframalpha.com/input?i=0.01+ms+at+speed+of+lig...

tcp_handshaker 4 hours ago | parent | prev | next [-]

>> I wonder if you can uncover where the data center is just by using ping command.

Not exactly, but you can uncover cloud providers like Google and Azure, who forget to tell you, their "availability zones" are in the same data center ;-)

jmalicki 3 hours ago | parent [-]

As long as they're on opposite sides so the same Shahed missile doesn't hit them both.

sophacles 4 hours ago | parent | prev | next [-]

You can find data centers by looking for hvac units in satellite photos.

hirako2000 3 hours ago | parent | next [-]

You can make a few phone calls. Maintenance is not avoidable until boston dynamics figure out how to clean the toilets at least.

Ekaros 4 hours ago | parent | prev [-]

Probably also grid connections like size of transformers and if there is prominent number of trucks going in and out.

jmalicki 3 hours ago | parent [-]

The relative lack of trucks is what would identify the data center. The only other buildings like that are warehouses, which have a lot more trucks going in and out relatively speaking.

hnlmorg an hour ago | parent [-]

In rural areas maybe. But plenty of data centres are in cities too. And European cities are not generally known for an abundance of parking spaces.

jmalicki an hour ago | parent [-]

The giant kilometer by kilometer hyperscaler data centers aren't in cities.

fragmede 4 hours ago | parent | prev [-]

That's some of how geolocation works. Ping can't go faster than the speed of light, so that gives you a circle for where something is. Ping from enough places and you can get a good enough idea, if you're the Iranian Guard or otherwise.

walrus01 3 hours ago | parent | prev [-]

You can be secretive all you want, but it's extremely difficult to hide massive heat exchanging systems and/or generators from aerial/space photography. Particularly at the scale of an AWS-like datacenter.

Building a fully camouflaged datacenter could be done at much greater cost, but you still can't hide its thermal emissions from infrared. Basically every watt hour used in a datacenter environment turns into waste heat ultimately rejected into the atmosphere (except for the 0.000000001% that leaves the facility as photons down a fiber), so if you have N megawatts of waste heat from a rectangular shaped building located on a 300 x 400 meter sized plot of land, it's going to stand out.

s1mplicissimus 3 hours ago | parent | next [-]

Wouldn't it be possible to pipe away the heat to the next city and use it as heating there? That way the heat emissions wouldn't be as noticeable

jmalicki 3 hours ago | parent | prev | next [-]

> except for the 0.000000001% that leaves the facility as photons down a fiber

Realistically you're getting photons returned too.

throwaway894345 2 hours ago | parent | prev [-]

Geothermal exists, but you would have to take care to design accordingly and even then there are plenty of other ways for a state actor to locate you. It probably doesn’t make much sense to spend money trying to hide from state actors; it’s probably better to (1) avoid conflict prone areas to the extent possible and (2) make it expensive for an attacker to shut you down (use more smaller data centers within a sensitive region, put some of them underground, etc) or (3) accept the risk of data center disruption.

logickkk1 3 hours ago | parent | prev | next [-]

"Stops billing" makes it sound generous. If those regions can't run customer apps, not charging for them is just the minimum.

pwarner 2 hours ago | parent [-]

IDK your S3 data may be fine, they're still incurring the cost to store it on those drives - even if they're buried in rubble /s

essefjo 21 minutes ago | parent | next [-]

There is already a S3 storage class for that: Amazon S3 Glacier Deep Archive

jabiko 25 minutes ago | parent | prev [-]

Also you have to remember the basics of statuspage messages: Its always just elevated error rates. Even when the error rate is elevated to 100%.

"We are observing elevated error rates when accessing objects stored in the affected region. Impacted customers may experience increased latency or intermittent failures while retrieving debris adjacent data." /s

nerdsniper 5 hours ago | parent | prev | next [-]

I'm surprised this reportedly only affected 19 server racks. Some of the small FPV quadcopter strikes I've seen videos of have collapsed entire homes. Even if the structure is more resilient than a fragile home, I would have expected the blast from a larger long-range drone like a Shahed to damage more server racks than that.

readams 15 minutes ago | parent | next [-]

Yea it's hard to reconcile such a small number of affected racks with such a widespread impact though, so this must not be the whole story. They're talking about a half a year to restore the data center. It must be more than a roof repair and 19 racks.

bluegatty 4 hours ago | parent | prev | next [-]

It's either 100lb or 100Kg, with a direct hit on a dense centre, it would damage a lot of racks, but if it's oblique, or indirect, impartial, the damage could be less pronounced. They could also be misrepresenting by diminishing the damage as there's a lot of information suppression going on.

downrightmike 4 hours ago | parent | prev [-]

The sheer amount of metal on every rack makes DCs very dense

neuroelectron an hour ago | parent | prev | next [-]

When I was working at AWS, which was a new service at the time, the example we often heard was a natural disaster or comet strike; would be what we were making our data centers redundant for. I don't think we were ever considered to be targeted during war and I'm sure they considered that they just didn't want to that affect that morale cost on the staff.

juliusceasar 3 hours ago | parent | prev | next [-]

Thanks 2nd Epstein War for all the fuck up in the world.

2ndorderthought 2 hours ago | parent [-]

It had to have been cheaper to just pay more in taxes then this outcome.

neuroelectron an hour ago | parent [-]

Not if you're rich and own representatives.

opengrass 43 minutes ago | parent | prev [-]

ZOMG free egress!?!?