That's a great answer, thanks. I'd missed that detail.

Honestly, a lot of those issues could be remedied with more optimized kernels, which LLMs could definitely write quite well now. I wouldn't be surprised if some deployments of Gemini Flash switched to diffusion for cases where there is low utilization, since Google showed you don't need to retrain an autoregressive model to rework it for diffusion inference.