The drop for K3.1 Pro & MiMo V2.6 Pro is quite surprising. Are they benchmaxxed?

Or, are we witnessing reverse-benchmaxxing? Meaning that first some entities let players optimise for a given benchmark, then they update the measurement methodology so as to retain some players on the top ranks, while dropping others. As former Italian prime minister, and highly controversial public figure, Giulio Andreotti once remarked "Thinking the worst is a sin, but you’re often right."

▲

prodigycorp 3 hours ago | parent | prev | next [-]

Transparently so. MiMo's training dashboard showed the DeepSWE hillclimb in real time.

▲

simianwords 3 hours ago | parent | prev [-]

I keep seeing that MiMo is in the frontier in terms of performance and cost but I see very few talking about it and the only ones that do talk about it are doing it in context of AA

▲

brainwad 2 hours ago | parent | next [-]

It's also dog slow at AA index tasks compared to other models in its class. Makes me think they optimised for getting on the intelligence/cost frontier at the cost of real world usability.

	▲	simianwords an hour ago \| parent [-]
		I’d bet on this

▲

wyre 2 hours ago | parent | prev [-]

I used MiMo for a couple of days waiting for my Codex sub to reset. I really liked it, its communication style was a breath of fresh air compared to how stale gpt-sol is.