When assessing probabilistic models the plots should be showing the mean a̶n̶d̶ ̶s̶t̶d̶e̶v̶ of many monte carlo simulations not just one line per model and claiming "look this model is more gooder!"

▲

memoriyato3 4 hours ago | parent [-]

standard deviation is misleading for non-standard distributions (fat-tailed, skewed, multi-modal, ...)

common mistake people make

	▲	FabHK 20 minutes ago \| parent [-]
		Not really. It's still the standard deviation, and it still gives you bounds on probability, for example the Chebyshev inequality: P(\|X-\mu\| > k \sigma) < 1/k^2. So, while for a normal RV, 5% of observations lie outside +/- 1.96 std.devs, for arbitrary RV (with finite variance) at most 25% of observations lie outside +/- 2 std.devs.