The evidence "actually supports the null" over what alternative?

In a Bayesian analysis, the result of an inference, e.g. about the fairness of a coin as in Lindley's paradox, depends completely on the distribution of the alternative specified in the analysis. The frequentist analysis, for better and worse, doesn't need to specify a distribution for the alternative.

The classic Lindley's paradox uses a uniform alternative, but there is no justification for this at all. It's not as though a coin is either perfectly fair or has a totally random heads probability. A realistic bias will be subtle and the prior should reflect that. Something like this is often true of real-world applicaitons too.

▲

_alternator_ 11 hours ago | parent [-]

Thank you. The main problem with Bayesian statistics is that if the outcome depends on your priors, your priors, not the data determine the outcome.

Bayesian supporters often like to say they are just using more information by coding them in priors, but if they had data to support their priors, they are frequentists.

▲

kgwgk 10 hours ago | parent [-]

If they were doing frequentist inference they wouldn’t be using priors at all and there is nothing frequentist in using previous data to construct prior distributions.

▲

uoaei 9 hours ago | parent [-]

Not true. In frequentist statistics, from the perspective of Bayesians, your prior is a point distribution derived empirically. It doesn't have the same confidence / uncertainty intervals but it does have an unnecessarily overconfident assumption of the nature of the data generating process.

▲

kgwgk 7 hours ago | parent [-]

Not true. In frequentist statistics, from the perspective of Bayesians and non-Bayesians alike, there are no priors.

—-

Dear ChatGPT, are there priors in frequentist statistics? (Please answer with a single sentence.)

No — unlike Bayesian statistics, frequentist statistics do not use priors, as they treat parameters as fixed and rely solely on the likelihood derived from the observed data.

▲

zozbot234 6 hours ago | parent | next [-]

There's always priors, they're just "flat", uniform priors (for maximum likelihood methods). But what "flat" means is determined by the parameterization you pick for your model. which is more or less arbitrary. Bayesians would call this an uninformative prior. And you can most likely account for stronger, more informative priors within frequentist statistics by resorting to so-called "robust" methods.

▲

kgwgk 2 hours ago | parent | next [-]

It’s not true that “there are always priors”. There are no priors when you calculate the area of a triangle, because priors are not a thing in geometry. Priors are not a thing in frequentist inference either.

You may do a Bayesian calculation that looks similar to a frequentist calculation but it will be conceptually different. The result is not really comparable: a frequentist confidence interval and a Bayesian credible interval are completely different things even if the numerical values of the limits coincide.

▲

zozbot234 2 hours ago | parent [-]

Frequentist confidence intervals as generally interpreted are not even compatible with the likelihood principle. There's really not much of a proper foundation for that interpretation of the "numerical values".

	▲	kgwgk an hour ago \| parent [-]
		What does “as generally interpreted” mean? There is one valid way to interpret confidence intervals. The point is that it’s not based on a posterior probability and there is no prior probability there either.

▲

_alternator_ 3 hours ago | parent | prev [-]

First, there is not such thing as a ‘uninformative’ prior; it’s a misnomer. They can change drastically based on your paramerization (cf change of variables in integration).

Second, I think the nod to robust methods is what’s often called regularization in frequentist statistics. There are cases where regularization and priors lead to the same methodology (cf L1 regularized fits and exponential priors) but the interpretation of the results is different. Bayesian claim they get stronger results but that’s because they make what are ultimately unjustified assumptions. My point is that if they were fully justified, they have to use frequentist methods.

	▲	kgwgk 2 hours ago \| parent [-]
		One standard way to get uninformative priors is to make them invariant under the transformation groups which are relevant given the symmetries in the problem.

▲

kgwgk 3 hours ago | parent | prev [-]

If you want to say that when you do a frequentist analysis which doesn’t include any concept of prior you get a result that has a similar form to the result of a completely different conceptually Bayesian analysis which uses a flat prior (definitely not “a point distribution derived empirically”) that may be correct. It remains true that there is no prior in the frequentist analysis because they are not part of frequentist inference at all.