Yeah, this is exactly what I thought when I read this post. It seemed like the author either hasn't worked in big tech, or hasn't worked in the industry very long. It's extremely likely that the engineer who designed this was standing on his desk shouting "it's going to cost THIS MUCH MONEY. I want to make sure that EVERYONE IS OK WITH THIS." and was met with shrugs.

Here's how a big tech reporting chain sees this situation when everything is smooth sailing: "We're growing 3x year-over-year? After 2 years, the cost will be an order of magnitude higher no matter what solution we pick. The constant factor doesn't matter that much. But we have such an incredible roadmap that we will book more than an order of magnitude of revenue, backed by this new ledger project. The cost will always be a nonissue because of growth."

And then 2 years go by, and this incredible product growth adds a bunch of ledger entries that weren't there 2 years ago, someone nudges your reporting chain with the question, "this is pretty expensive.. what gives?" and then someone with a good combination of social and technical skills points out that a migration to your existing storage solution would be a cost effective way to continue growing.

At every step of the way, everyone is generally happy with what's going on.

Also totally possible that it was just an unpublished partnership of sorts between AWS and Uber. AWS wants the logo and a big case study implementation to give the product some credibility or a boost. Uber may not have been charged at all, may have even been paid to use AWS. The Uber developer may not have even known, just was given an edict to build it on dynamodb.

I think it's important for leadership to clearly define what right is in these cases, too, otherwise, you get as many ddefinitions of "right" as you have people, times, and places.

Easy to say, but it's a real human cost to relying on people to figure out what you mean rather than explaining what you mean. Not enough time is spent on cultivating effective communication and training. Everyone wants everything done yesterday and don't feel like investing in their own people.

ERP implementations probably don't fail for those kind of architectural reasons?

> Do you think that the social climbers who approved these obviously crappy projects learned anything?

Just the framing of this question makes it seem like you simply don't like people in management / decision-makers, and you want something bad to happen to them. Maybe that's wrong, hopefully it is, but the rest of the comment doesn't do much to dissuade me of that impression either.

I've certainly learned a great deal from my own crap glassy-eyed decisions throughout my career.

Right - if your engineering organization ships designs that are bad economically, the solution is to introduce a culture of predicting costs before committing to a design, and processes to help enforce that culture.

Add "expected budget, double-checked by at least one other principal engineer" to the project checklist.

Have the person most responsive for the $8m "mistake" be the person to drive that cultural change, since they now have the most credibility for why it's a useful step!

Letting interns carry six figure equipment, which would also be unexpectedly heavy especially if this happened some years ago, would be a weird thing for any lab I’ve worked in. There are too many things that can predictably go wrong in the hands of an inexperienced person, as happened here.

Interns wouldn’t even be allowed to use $100K VNAs without a lot of supervision because so many things can go wrong. Damaging one of those small precision connectors is easy to do and can be a costly repair that brings delays to the lab, and that’s before you even start making measurements.

I wonder if part of the offense was that the intern was breaking protocol by moving the equipment. Alternatively they probably failed to explain the rules and expectations to the intern. Or maybe some lazy engineer tried to pawn off their work on to an intern without thinking about the consequence.

I cannot, of course, speak about this particular incident, but a person inclined to skip procedures expressly implemented to avoid the problem which occurred, or who ignores clear warnings that a problem is developing, is a liability, not a trained asset.

> But on the other hand, I'd probably fire the manager/executive responsible for that move, rather than the individual developer who probably suggested it.

And you just teached all your workers to be as cautious as being freezed, never be proactive, keep the status quo as much as they can, avoid being noticed, and never take a step without being forced or having someone else to take 100% blame (with paper trail) if things go south.