It is a good post, but is missing the connection to continuous normalizing flows. Diffusion models, flow matching, consistency models are biased approximations of continuous normalizing flows (which themselves have some slight biases, but less). Adversarial losses can somewhat help with bias (e.g. RL, GANs), but training those has issues.

As explanation, something I wrote previously:

The most common approach to modeling continuous distributions is to train a reversible model f that maps it to another continuous distribution P that is already known. The original image can be recovered by tracking the bits needed to encode its latent, as well as the reverse path:

  −log P(f(x)) − log|det ∂f/∂x (x)|

  dx/dt = g(x)

  log|det ∂f/∂x (x)| = ∫ Tr(∂g(x)/∂x) dt = ∫ E_{ε∼N(0,I)} [εᵀ ∂g(x)/∂x ε] dt

  dx′ = g(x′)dt + ε(t)dW

  d(δxᵀδx)/dt = 2δxᵀ ∂g(x)/∂x δx,

I briefly covered that connection in an earlier blog post: https://sander.ai/2023/07/20/perspectives.html#flow ... but it's definitely something that might deserve a longer-form treatment at some point :)