Remix.run Logo
▲ cubefox 18 hours ago

What's that?

▲johntb86 17 hours ago | parent [-]

https://arxiv.org/abs/2602.06036 - they use a faster (single-pass) diffusion model to generate predictions for 7 to 15 tokens in a single step. Then if they're accepted that really speeds up the autoregressive model because it doesn't have to generate tokens one at a time.

▲cubefox 7 hours ago | parent [-]

I'm pretty sure fast speculative decoding models need to have the same objective (NTP, MDM, NCM) as the larger model they try to accelerate.