Remix.run Logo
▲ vova_hn2 3 hours ago

I think that the fact that TFA doesn't mention speculative decoding at all is not a good sign. It casts doubt both on the accuracy of the data and on the competence of whoever collected it.

I can imagine that in some cases the acceptance rate for non-English responses may be much higher, because while words on average need more tokens to spell, it is more "obvious" what the next token should be, so it is easier for the speculation model to predict it accurately. Information density per token is lower.

Or, maybe the acceptance rate is much lower, if the speculation model is tuned for English and doesn't work well with other languages.