| ▲ | jwolfe 4 hours ago | |||||||
That's versus Opus 5.5 max effort, no? Opus high has the closest cost and intelligence characteristics and generates far fewer output tokens per task than the one Kimi 3.1 Pro effort level we have data for. | ||||||||
| ▲ | thepasch 4 hours ago | parent | next [-] | |||||||
Yes, max effort is what the displayed-by-default score on the index is measured by. At xhigh, Opus 5.5 outputs 100M tokens, but remains only 1 point ahead of K3.1 rather than 5. And Sonnet at xhigh outputs 110M tokens and trails K3.1 by 3 points rather than leading by 3, at the same price point per mtok. Opus 5.5 high trails K3.1 by 1 point, but outputs only 53M tokens, so it indeed comes out cheaper per-task. I think it's still remarkable that this is effectively an open model that matches the frontier for the first time in a long time, pretty much since DeepSeek R1. | ||||||||
| ||||||||
| ▲ | TobTobXX 3 hours ago | parent | prev [-] | |||||||
I think that this is Kimi K3.1 Pro on max too. Unfortunately, AA doesn't seem to test for other effort levels for models not from Anthropic/OpenAI. | ||||||||