> 3. Had the higher-priority request arrived even one millisecond earlier or later, the update would have completed normally.

Well, that's comforting to know.

BBC: "Flight chaos caused by software defect in space of a millisecond, report says"

Sky: "'Millisecond' software error caused air traffic outage that grounded thousands of flights"

The Guardian: "Flight chaos for hundreds of thousands was caused in ‘millisecond’ by software error"

Sounds like pure bad luck.

▲

cmpxchg8b an hour ago | parent | next [-]

Safety critical systems demand formal verification. This wasn't bad luck, this was poor craft.

▲

philipwhiuk 2 hours ago | parent | prev [-]

Or insufficient testing?

▲

nradov 2 hours ago | parent | next [-]

Testing isn't an effective way to catch most race conditions. Code reviews, static analysis, and rigorous enforcement of concurrent coding standards is usually a better approach.

	▲	jiehong an hour ago \| parent [-]
		Perhaps something like what TigerBeetle does: deterministic simulation testing.

▲

dtf an hour ago | parent | prev [-]

Or maybe even just reviewing logic that is subject to pre-emption.

Maybe I'm being too harsh.. on the plus side the system has at least failed hard every time there's been a fault. Nobody has died. But it's been 3 times now in the past couple of years, and two of those times resulted in over 2000 flights cancelled and days of backlog, and misery for hundreds of thousands. It's really not acceptable.

	▲	chrisjj an hour ago \| parent [-]
		> the system has at least failed hard every time there's been a fault You don't know that.