I am more familiar with Bayesian than frequentist stats, but given that they are mathematically equivalent, shouldn't frequentist stats have an answer to e.g. the loss function of a VAE? Or are generative machine learning inherently impossible to model for frequentist stats?

Though if you think about it, a diffusion model is somewhat (partially) frequentist.

I guess you have me thinking more... things like Parzen window estimators or other KDEs are frequentist...

But while it's a probability distribution, to a frequentist they are estimating the fixed parameters of a distribution.

The distribution isn't generative, it just represents uncertainty - and I think that's a bit of the deep core philosophical divide between frequentists and Bayesians - you might use all the same math, but you cannot possibly think of it as being generative.

They do!

https://arxiv.org/pdf/2510.18777

But that doesn't mean a frequentist views a VAE as a generative model!

Putting it another way, Gaussian processes originated as a frequentist technique! But to a frequentist they are not generative.