We've got a coin here! Let's ask a frequentist and a bayesian what they think about the probability that flipping it would make it land heads up?

Frequentist: How would I know? I haven't seen it flipped once, nor do I know how you've selected it from all the other coins that exist.

Bayesian: It's 50%!

Then we flip the coin 10,000 times and observes that it landed up heads exactly 5,000 times.

Frequentist: Huh, that's weird. You see, if the probability was 1/2, the expected deviation from the mean in this case would've been 50, so I'd expected to see either about 4'950, or 5'050 heads... still, MLE provides the answer of 1/2 bu-u-ut...

Bayesian: It's 50%!

This two-strawmen thought experiment clearly demonstrates the superiority of the Bayesian approach in learning useful information from the real-world observations.

It's really kinda shame that both personal certainties and physical probabilities follow the same algebraical rules while having entirely different nature; most of the time, you are not very interested in how much is someone is certain of some outcome, you're much more interested in the actual outcome or at least the actual probability of that outcome. Granted, most of the time you can only readily access someone's certainty of an outcome, but this is just a proxy for the quantity you're actually interested in knowing.

> You are better saying that the probability distribution of the probability is β(65,35) or maybe β(65.5,35.5) or β(66,36)

s/probability/your personal certainty/g. The probability of the coin landing heads up is what it is, and it usually doesn't depend on any of your knowledge.