I dont know how one can objectively evaluate forecasting. We observe just one universe. We overfit our models to this one observation we have.

Which is why at the end of the day why nobody trusts or blames the forecasting models, specially in all or nothing types of decisions.

▲

sh34r 3 hours ago | parent [-]

This is the pseudo-intellectual version of your boomer relative ranting about the weatherman on TV, “how does this guy still have a job when he’s always wrong!”

Or the poll-bashing that happened after the 2016 election when an outcome with a 29% chance per one model was observed.

Any interesting forecasting problem is underconstrained. Monte Carlo methods are the classic way to mitigate that one-universe problem.

Nobody can predict the future with certainty. All models are wrong, but some are useful.

▲

whatever1 3 hours ago | parent [-]

How does Monte Carlo help mitigation?

I fully agree that all models are wrong.

Their main usage is just replaying the past or encoding the modelers assumptions for the future.

But there is no way to know if a model is good or bad.

There would be if we had infinite universes.

	▲	tom_mc 19 minutes ago \| parent [-]
		Monte Carlo methods are used to estimate the risk associated with a prediction (and presumably an action taken against that prediction). Does that answer your question? I can elaborate if you like.