That’s because diffusion models struggle with memorizing sequences while it’s essentially what autoregressive models excel at.

I’ve tried to fine tune a diffusion model to memorize an MD5 hash and after days of hacking on it, it failed to completely memorize it. Autoregressive models of any size learn it over <1000 iterations.