| ▲ | aqme28 4 hours ago | |||||||||||||
Do you have any evidence that this model is bench-maxxed? I know that's particularly difficult to quantify. If there is an indicator of bench-maxxing, that just becomes the new benchmark to benchmax. | ||||||||||||||
| ▲ | deaux 3 hours ago | parent | next [-] | |||||||||||||
Here, filtered down for you. [0] Look at the individual benchmarks, not the combined one. You can tell that this model is much more benchmaxxed as its relative ranking swings between benchmarks is much larger. This is a hallmark. [0] https://artificialanalysis.ai/models/qwen3-8-27b?models=deep... | ||||||||||||||
| ||||||||||||||
| ▲ | achrono 3 hours ago | parent | prev [-] | |||||||||||||
Sounds obvious but just try using the models for anything outside the evals. Take something arcane from Greek history, use it to create a masked linguistic puzzle, which you then ask the model to solve mathematically, all wrapped as an ask to generate ASCII art. Yes, all these elements exist in some form in the evals but the key is in how utterly unconventional the elements are that you pick and in how you combine them. I have consistently noticed Opus 4.8 and GPT-5.6 far outshine the Chinese models. Gemini is sort of middle of the road, Grok is better than Gemini but not really close to Opus/GPT. OAI & Anthropic still remain unbeaten by a wide margin in my eyes. | ||||||||||||||
| ||||||||||||||