| ▲ | mridulmalpani 7 hours ago |
| Gemini 4 Argon is a new leader in Pareto-frontier. Looks like, Google is back in the race. https://arena.ai/leaderboard/agent/chat/pareto For almost same intelligence (+15.56%, +15.18%), drastic difference in cost/task ($3.51 vs $0.41) |
|
| ▲ | ianm218 4 hours ago | parent | next [-] |
| Why do these benchmarks often show models running on Max i.e. Opus 5.5 (max) is on the frontier? I would think it is never really economical to run max |
| |
| ▲ | jonas21 2 hours ago | parent [-] | | By definition, the model with the best performance will be on the pareto frontier (regardless of cost). | | |
| ▲ | seizethecheese an hour ago | parent [-] | | No. Pareto frontier is cost performance tradeoff. | | |
| ▲ | cloudbonsai an hour ago | parent [-] | | What jonas21 is stating is correct. If you have the absolute best performance, you are automatically on the Pareto frontier because you’re the most cost effective one (or the only model) on that performance tier. | | |
| ▲ | seizethecheese 31 minutes ago | parent [-] | | Huh, I guess you're right, but I interpreted jonas21's comment in relation to the above comment, which was about all models running on Max (which implies this would be a comment relating to all points on the pareto curve). |
|
|
|
|
|
| ▲ | cheema33 3 hours ago | parent | prev | next [-] |
| > Gemini 4 Argon is a new leader in Pareto-frontier. Looks like, Google is back in the race. For chat, yes. But switch to code on the same page, and the picture changes. OpenAI takes the lead. |
|
| ▲ | BrokenCogs 5 hours ago | parent | prev | next [-] |
| Every release I see a new benchmark with a new frontier. These graphs are just meaningless now |
| |
| ▲ | abeppu 5 hours ago | parent | next [-] | | In particular, where cost is the x axis, a well-funded company with a good but not best model can discount their new model until it reaches the frontier. In some sense this is as much a statement about the efficiency of markets in the presence of public measurement-based comparison and public list pricing as it is about research pace. | | |
| ▲ | kridsdale1 4 hours ago | parent [-] | | While true, Google genuinely has invested the most in computer and energy efficiency for LLMs. |
| |
| ▲ | yipinwong 5 hours ago | parent | prev [-] | | you can give it a meaning if you see the relavivity between models. no need to disregard the stat at all. |
|
|
| ▲ | vessenes 4 hours ago | parent | prev | next [-] |
| Muse reminded me we should have a time based chart as well - this year I hope we will begin to see time to completion stats at various intelligence tiers. The waiting makes a lot of things more tiring. |
|
| ▲ | 6 hours ago | parent | prev [-] |
| [deleted] |
