Remix.run Logo
▲ cube00 a day ago

> The workflow on election night, from my couch: edit some JSON, VPN into the NBC network, SSH into the server, git pull from master, pm2 restart all, text the producer that it's updated — then watch my change go out on air a few minutes later through Hulu. Doing that live in an emergency and seeing it on television minutes later is a pretty cool feeling.

Less "cool" and more asking for trouble if you make one misplaced comma, quote, or curly brace. The entire apps crashes and shows nothing to the broadcast audience or worse an error screen they'll screencap and meme.

▲dylan604 a day ago | parent | next [-]

> Doing that live in an emergency and seeing it on television minutes later is a pretty cool feeling.

I once heard a story of the slomo camera used during MLB games stopped working during a World Series game which meant that a huge feature of the broadcast was going to be unavailable. One of the engineering crew took it upon himself to look into it. Apparently, another camera angle caught the extra person at the camera during the broadcast which was seen by a high ranking executive who promptly called the director to see what was going on. His response boiled down to "someone is either getting fired, or getting promoted". A few innings later, the slomo feature was again available for the broadcast.

Sometimes, you just got to "get 'er done" on the air.

▲coffeefirst a day ago | parent | prev | next [-]

True but on election night you’re just changing things from null to one of two strings.

I’ve spent 17 years in news and it’s not uncommon to operate short term journalistic stuff off a Google sheet or xml file. It’s basically fine.

▲shermantanktop a day ago | parent [-]

If you know, you know. Lots of bystander comments in this discussion, finding fault or claiming they’d do it better. Maybe NBC News should be recruiting from here.

And all that is missing the real story, which is how one guy made tech magic happen, and how it feels to be praised for a technical accomplishment when those praising have very little idea what it really took.

▲thakoppno a day ago | parent | prev | next [-]

That part stood out to me too. However this article was from 2018, not last week it seems to me.

We did live debugging like this at the time, wouldn’t today.

▲ejfox a day ago | parent | next [-]

agreed, this is an artifact, I would do it very differently today

▲jauntywundrkind a day ago | parent | prev | next [-]

It sure seemed like there were a bunch of moments of live updating? Discussion with Chris (producer), saying, ok, it's updated now, try this. Yes a lot was data updates but there were a bunch of other times it seemed more systems code etc.

▲mkehrt a day ago | parent | prev [-]

I'm baffled by the claim that live debugging of this sort was ok in 2018. That wasn't that long ago. In 2018 we definitely knew this was a terrible idea.

▲thakoppno 5 hours ago | parent [-]

The stack described in this article would not have been extremely stable if memory serves. It seems likely to me the setup is one server, one client. In the live TV setting described, these sort of tactics seem like a decent trade. There’s a director of the show who can work around any technical problems encountered on the big board.

▲kdazzle a day ago | parent | prev | next [-]

Maybe they have the CI even if not the CD…?

▲afavour a day ago | parent | prev [-]

Yeah I hope that’s a simplification. At the very least I hope some automated testing is scanning that code change in a PR before it gets pulled down to the server.

> worse an error screen they'll screencap and meme.

Or even worse, you misattribute a result to the wrong candidate, it gets screencapped and used as evidence in another “election fraud” conspiracy theory.

I actually think the safety guards they’d add for software like this would be the more interesting part of the story!