Remix.run Logo
0xbadcafebee 2 days ago

Don't let perfection be the enemy of the good. In industrial testing automation, you often don't test every feature completely. You calculate the cost-benefit of the complexity, time and cost it takes to test features, and target the most valuable ones. (I used to write automation test suites for industrial appliances with hundreds of thousands of permutations)

At the very least, you can test one single feature end to end, just to get a baseline that nothing is massively, obviously broken (like it was in this case!). This lack of fundamental testing is what caused the CrowdStrike bug that brought down Delta Airlines. This kind of test is extremely easy, quick, cheap, and simple, and catches the worst bugs. In other words, it's the most valuable test.

Companies will continue to let these bugs happen until we mandate a minimum amount of fundamental testing before release.