This might be the first time I’ve favourited a post without upvoting it.

This is really sus. I’d love a more detailed explanation and “take” from AA on what problem they felt needed correcting and why things have shuffled the way they have. And how they can claim their index has relevance if positions can be switched around so easily.

small (700b) open-weights models are great at coding and tool calling but lack world knowledge. because the parameter difference is so huge T to B, of course they also are not as intelligent.

this is i think the difference in this new analysis.

doesnt mean deepseek is bad or useless.

From my rough recollection, all 3 of the past updates all had the effects of decreasing the score of open weight models.

One was rushed and same-day updated as GLM 5.3 (iirc) got added; of course its main change to rankings was lower GLM scores.

Fun fact: By changing the weights of the input you can change the output as you want.