The real caveat is that 538 was a Monte Carlo model, and is only as good as its inputs. "Here's what the current spread in polling numbers is *given our model and the current polling and their reported uncertainties.*" Polling uncertainties are themselves computed under certain models, and those models are subject to errors. I don't think 538 hid this, but it's a difficult caveat for people to reason about because the sorts of modeling errors that have the most influence usually represent "unknown unknowns".

That's a core mechanic in games like Dispatch.

People don't like seeing a 95% chance of winning and then losing. The game tweaks the odds, so certain thresholds become gimmes (something like "if the displayed odds are better than 75%, treat them as 100%").

> Discussion of stats models is always complicated by the fact that a lot of people will read "30%" as a "no" prediction and claim your model is wrong if the thing happens.

I've even heard things like "70% chance of Hillary winning means she gets 70% of the votes!" (and tangentially, my far-too-long argument with someone on Reddit who insisted "there is no way in hell 50% of the people in this town make above the median income"...)

> it turned out that predicting elections is still very hard

So maybe we shouldn't be doing it. The value of predicting an election in the large out in public seems kind of dubious, and it's more like gambling than actually being useful. A candidate only runs, and continues running, if they think they can win. All predictions like these do is confuse voters leading up to election day and while they are voting. It keep candidates from making strong cases for their platform, keeps the voters from listening to the candidates' platforms, and encourages team-based partisan politics.

> FiveThirtyEight performed only slightly better than what you could find in any other reputable newspaper

FiveThirtyEight gave Trump double the odds of the next highest reputable prediction, which was The New York Times Upshot (15%). Princeton Election Consortium gave Trump less than 1%.

That is not "only slightly better" to anyone who's statistically literate.

A FiveThirtyEight reader in 2016 was significantly better calibrated regarding Clinton’s chances than a reader of other reputable newspapers.

538 was never about magically making polls more reliable, and only people that don't understand what polls are could think that (caveat: lots of people don't understand how polls work).

538 was about analyzing and communicating the information from those polls in an easily accessible form. If you came to the site for that, you weren't mad that they "predicted poorly something that was impossible to predict from the data sources they used" ... you were just mad at Trump for winning (despite polls suggesting otherwise).

Polls became much less interesting as an Entertainment category once we all had experience with how unreliable they are.

Yeah, Nate has talked a number of times about polling errors being correlated across states. In fact that's probably one of the most common mistakes models can make, treating correlated inputs as independent. There's a long history of that mistake in financial markets as well.

In 2024 the single most likely outcome his model had was trump winning all 7 swing states. The second most likely was Harris winning all 7.

They had him at 30% at a time when most reputable media had him in single digits at best. You might not remember now, but the result of that election was a real shock.