Remix.run Logo
mcv 3 days ago

Yeah, at some point the article says:

> I’d done everything right. Vault encryption keys stored separately from my main infrastructure. Defense in depth. Zero trust architecture. The works.

Did you? Is putting all your eggs in one basket "defense in depth"? Is total trust in AWS "zero trust architecture"?

I'm not defending AWS here; they fully deserve all the fallout they can get from this, and I do feel for the dev who lost all their stuff through AWS's fuckup. Lots of people do the same.

My current employer does the same. It's a major bank, and all of their stuff is Microsoft. Azure, SharePoint, Office, Teams, the works. I think it's foolish to trust a single foreign company with all your vital data and infrastructure, operating in q country where the government demands access to everything, but this is what everybody does now.

We trust "the cloud" way too much, and expose ourselves to these sort of fuckups.

seuros 3 days ago | parent [-]

I dont disagree with your broader point—centralizing everything in one provider is a systemic risk.

The architecture was built assuming infrastructure within AWS might fail. What I didn’t plan for was the provider itself turning hostile, skipping their own retention policy, and treating verification as a deletion trigger.

throwaway413121 3 days ago | parent | next [-]

> The architecture was built assuming infrastructure within AWS might fail.

From what i gather it was not. Or did you have a strategy for a 0-warning complete AWS service closure? Just imagine AWS closing their doors from one day to the next due to economic losses, or due to judicial inquiries into their illegal practices: were you really prepared for their failure?

The cloud was never data living in tiny rain droplets and swimming across the earth to our clients. The cloud was always somebody else's computer(s) that they control, and we don't. I'm sorry you learnt that lesson the hard way.

darkwater a day ago | parent | prev | next [-]

If you have very important data for you, and you don't pay very high bills to AWS, you should really have at least a cold backup somewhere else (even on your own hardware). If you have a big cloud account, paying big money every month, at least with AWS you are in a pretty safe spot, even if people will say a different thing here.

And if you have a similar horror story with a tens/hundred of thousands of dollars (or more) monthly AWS invoice, please speak, I'm very curious to learn what happened.

mcv 3 days ago | parent | prev [-]

It probably wasn't even hostility; it's just that accounts is also an infrastructure component. And when that fails, everything fails. Tying everything to a single account creates a single point of failure.

It's one of the reasons I don't use my Google account for everything (another is that I don't want them to know everything about me), and I strongly dislike Google's and Microsoft's attempts to force their accounts on me for everything.