Discussion of stats models is always complicated by the fact that a lot of people will read "30%" as a "no" prediction and claim your model is wrong if the thing happens. On the one hand, one strategy is to "hide" the numbers a bit behind a blaring headline that says "we are not sure!!" It's a bit of an art to decide when to be "sure" or not. On the other hand, in research for example you can just say screw it, I care if the correct people are correct, not if a bunch of wrong people are wrong.

I feel like the correct strategy for 538 when it was actually niche was to be precise, but then it went viral and maybe should've hit the IDK button much harder and more often after that.

▲

gh02t 6 hours ago | parent | next [-]

The real caveat is that 538 was a Monte Carlo model, and is only as good as its inputs. "Here's what the current spread in polling numbers is *given our model and the current polling and their reported uncertainties.*" Polling uncertainties are themselves computed under certain models, and those models are subject to errors. I don't think 538 hid this, but it's a difficult caveat for people to reason about because the sorts of modeling errors that have the most influence usually represent "unknown unknowns".

▲

this_user 3 hours ago | parent | next [-]

Building a model for predicting the ultimate winner of a US presidential election is particularly difficult, because you are dealing with noisy input data and nonlinear effects, i.e. just a few thousand votes in a few key states can completely flip the outcome. If you then have poorly calibrated polls with a large margin of error, there is really nothing much you can do.

On the other hand, it does raise the question how valuable the 538 models for something like this really are if the outcome is a coin flip anyway.

	▲	ngriffiths 2 hours ago \| parent [-]
		Exactly, and correlated errors, where a polling error in one state predicts similar errors across the board. I disagree that it's all pointless though. Most basically it's smart for campaigns to have a good model and let that inform strategy where appropriate. Since the president is a big deal other people's decisions are also impacted, and in the long run it pays to have good predictions of those chances. Also, the outcome sometimes is fairly certain and that isn't always easy to see.

▲

bryanlarsen 3 hours ago | parent | prev [-]

Regularly referring to that ~30% spread as "one polling error" made this a lot more understandable than most statistics for many people.

▲

bsimpson 5 hours ago | parent | prev | next [-]

That's a core mechanic in games like Dispatch.

People don't like seeing a 95% chance of winning and then losing. The game tweaks the odds, so certain thresholds become gimmes (something like "if the displayed odds are better than 75%, treat them as 100%").

	▲	tantalor 5 hours ago \| parent \| next [-]
		That's stupid. That would piss me off.
	▲	a_t48 4 hours ago \| parent \| prev \| next [-]
		Fire Emblem does something complex with averaging random numbers to do the same thing - a 95% chance to hit becomes 99.5, and the reverse for low percentages.
	▲	lmm 2 hours ago \| parent \| prev [-]
		Conversely weather forecasters report a 40% chance of rain when the actual chance is 10% or similar. So I have a bit of sympathy for people who don't have a good intuition for probabilities, given that the world is constantly gaslighting them.

▲

FireBeyond 4 hours ago | parent | prev [-]

> Discussion of stats models is always complicated by the fact that a lot of people will read "30%" as a "no" prediction and claim your model is wrong if the thing happens.

I've even heard things like "70% chance of Hillary winning means she gets 70% of the votes!" (and tangentially, my far-too-long argument with someone on Reddit who insisted "there is no way in hell 50% of the people in this town make above the median income"...)