Mission critical systems that are in use 24/7 are not managed based on week day vs weekend shifts. Or at least shouldn't be. My (hopeful) assumption is that air traffic control isn't based on the "weekends are the off hours" mindset.

I did web dev work for an airline and we couldn't even release new updates to the loyalty benefits website during a weekday (even though there was no connection to operations or flights). The fact that this company would reboot a critical comms server at 3 pm on a Friday is indeed bizarre.

It should be fine to update at any moment. Why? Because systems like this ought to be hot spare or blue/green models that have a full working copy at any given time, and a failover/failback.

Maybe patching a critical vulnerability? I’m also at a loss for why they chose Friday afternoon.

A security patch would be a reason not to wait.

Rolling out updates on a Friday afternoon is still practiced in various industries.

> Anybody familiar with these systems know why they would do an update like this at 3pm on a Friday

As an insider to the technology industry, I would surmise the technical reason for it is that they didn't want to work over the weekend.

(Yes, this comment is tongue in cheek, but not everyone follows the "don't deploy on Friday" rule, even people in mission critical fields.)

This is why you don't push PROD on fridays...

Nothing like a good Friday afternoon did lot to ruin a weekend.

Perhaps.

But the entire first months of the Iran War was started on Friday after the markets closed. It took a few weeks for most Americans to catch on.

My money is on some mundane ops issue like "it takes lots of time to get sign off for the upgrade from lots of different stakeholders and the necessary upgrade was falling days or weeks behind, leading to a key IT person to decide to YOLO the upgrade/deployment".

I've heard that Sec Transport Duffy has been pushing FAA upgrades faster than is safe, so maybe the employee rushed it so this upgrade wouldn't hold back some other larger timeline. There have been several articles recently about how the FAA is full of people who are "too risk averse", suggesting that there might be an effort to push riskier projects faster. Having a nationwide outage is one of the likely symptoms of raising risk tolerance.

[dead]

Collins/ARINC is used for oceanic HF radio. They run "San Francisco Radio" and "New York Radio" specifically, which are the HF radio operators that relay oceanic messages to ATC.

(They also own a small number of private VHF stations for relaying company messages to air carrier operations domestically, but these aren't used for ATC purposes.)

Otherwise, the VHF radio links are operated directly by FAA TechOps.

L3Harris owns the telecommunications backhaul between facilities.

Cue that line from DS9 where O’Brien says he’d never be caught without a secondary backup during a fight, and the cardassian is like “you have a second backup?” incredulously. https://youtu.be/o7o9ZtipVyo?is=y3for_WXwISmuqpd

I wouldn’t say anything but this is approximately the only time in my entire life this scene will be relevant.

The more time I spend working in tech, the more I realize that everything important is held together with cardboard and duct tape.

Why only “one” network at all!?

The Internet was purpose designed to tolerate disruption as major as a nuclear war, which it achieves through decentralisation.

Private networks exist to make idiot managers feel more confident while actually dramatically reducing availability by putting all their eggs into a single basket.

Yeah - like, why do they not have an "HA" cluster, where they patch and reboot a subset at a time... Something this critical should have some sort of failover/load-balancing in-play...

Maybe the systems connected to it are not properly designed to handle such a reboot

> Apparently the L3Harris CEO was ousted in August in a Jensen-like scandal.

You mean he facilitated sale of restricted AI chips to China?

Sorry, but I am tired of the effort to make it look like Jansen's crime was infidelity.

If he just went to whore house he would be fine (well, ignoring a possible expensive divorce).

Especially Friday the 13th.

well is there a rollback procedure?

and they said "wont deploy on friday? thats 20% of the week in lost productivity!"

This indeed is a poor choice of words…

Isn’t Friday usually a light travel day?

People who have a testing window Saturday morning, which is how a lot of legacy industries with cyclical schedules work. Weekends are slower air travel days.

A junior engineer according to my table: intern: break the build and go home. junior engineer: break the build and go home for the weekend. staff engineer: break the build and go on vacation. senior engineer: break the build and retire.

There are no weekends in the airline industry, in case you didn't know.

Everything operates normally, as any other day.