We do use spec decode on all these models (I don't know if the paper is using that but I'm assuming not given the models listed) but not all these models have integrated spec decode stats in the responses. Turns out GLM 5.3 does though so here is a pass on that model.

   Language      Accept   Tokens/step   0/3 steps   3/3 steps   Output tok/req
   English       76.53%      3.296         9.4%        62.8%            549
   Chinese       73.37%      3.201        11.3%        58.2%            754
   Russian       73.88%      3.217        11.1%        58.5%            739
   French        75.50%      3.265         9.8%        60.9%            631
   S. Pashto     70.44%      3.113        12.9%        54.5%           1332
   Tigrinya      79.72%      3.391         8.2%        67.8%           3643
   Shan          75.64%      3.269        11.2%        63.0%           3675
   Tibetan       74.67%      3.240        10.3%        60.2%           2857
   Overall       75.75%      3.273        10.3%        62.2%           1772

▲ pama 4 hours ago | parent [-]

Thanks. So the acceptance rates are rather similar across languages.

▲ scottcha 4 hours ago | parent [-]

Yes it looks that way. I pulled the energy data for GLM 5.3 as well just to check the core findings are roughly the same and they do seem to be so spec decode doesn't seem to have the same divergence as the prefill/decode energy (which is rougly a function of thinking/decode lenghts and less a property of individual tokens).

  Language      Strict  Lenient   Cap   Tok/req     J/req    J/tok   Req x EN  Tok x EN  J/tok x EN
   English        100%     100%     0      548.7    230.33   0.4198       1.00      1.00         1.00
   Chinese         96%      96%     1      754.0    694.47   0.9210       3.02      1.37         2.19
   Russian         96%      98%     2      738.8    592.15   0.8015       2.57      1.35         1.91
   French         100%     100%     0      631.0   1101.84   1.7462       4.78      1.15         4.16
   S. Pashto       88%      98%     6     1332.1    208.06   0.1562       0.90      2.43         0.37
   Tigrinya        56%      62%    19     3642.9   1445.84   0.3969       6.28      6.64         0.95
   Shan            44%      54%    27     3674.8   3791.59   1.0318      16.46      6.70         2.46
   Tibetan          2%      70%    15     2856.9   5306.68   1.8575      23.04      5.21         4.42