Remix.run Logo
▲ CuriouslyC 3 hours ago

To simplify the explanation further, the main difference in practice between energy based and more traditional models is that they throw out normalization.

▲xgk 3 hours ago | parent | next [-]

That's also my default assumption: P(x) ∝ e^(−E(x)) Or, in more detail,

    E(x) = −log P(x) + constant
If that's the correct interpretation, then the main advantage of energy-based methods over strictly probabilistic methods seems to be that, since the energy function does not need to integrate to one, it can be parameterised with any nonlinear regression function. That means density estimation is basically a nonlinear regression problem.

This is the interpretation in [1]. I am not sure, however, if the paper here is an unnormalised probabilistic model.

[1] Y. Song, D. P. Kingma, How to Train Your Energy-Based Models. (2021) https://arxiv.org/abs/2101.03288

▲datsci_est_2015 2 hours ago | parent | prev [-]

Which can be a huge advantage, as normalization is an often-overlooked source of bias in modeling.