Remix.run Logo
ssivark 3 hours ago

My personal opinion is that statistics textbooks usually come from a prescriptive perspective, and that makes it challenging for the reader to get visceral intuition for what is actually going on. Any reader would be far better off just visualizing the damn distribution / samples and using reasonable judgement, instead of implicitly assuming a Gaussians distribution and blindly memorizing tests / formulae. Making the distributions explicit allows us to model them and get an intuition for what the samples are telling us. I would whole-heartedly recommend the Model based machine learning book to anyone (online version is free) https://mbmlbook.com/

eru an hour ago | parent [-]

How do you make 'reasonable judgements'? How do you tell whether someone else made reasonable judgements? How do you judge other people's intuition?

Modelling distributions explicitly sounds nice, yes.

ssivark an hour ago | parent [-]

Look at the histogram and think about what distribution one could reasonably impute from samples. And what you would set as bounds for "outliers", per your needs. While we're at it, let me also say that it might be useful to specify outlier bounds not just based on the spread in sample values, but the costs/payoffs they imply for your application.

If you are not doing something crazy, most reasonable people would agree with your judgement. Conversely, if you are making non-obvious inferences where reasonable people disagree, you are in murky water and no sophisticated statistical method will save you. Math is not magic; theorems merely recycle (launder) modeling assumptions into results.