| ▲ | seizethecheese 2 hours ago | ||||||||||||||||
Does anyone really believe the cost data for benchmarks? Pretty sure all models will behave differently at this point when they get benchmark shaped questions. (And I’m sitting on data showing Sol 6.1 and Qwen 3.8 Max cost roughly the same per task, for real tasks.) | |||||||||||||||||
| ▲ | eli an hour ago | parent | next [-] | ||||||||||||||||
The cost per task metric is accurate for those benchmark tasks. If your tasks don’t look like benchmark tasks..you should use your own benchmarks :) | |||||||||||||||||
| |||||||||||||||||
| ▲ | killingtime74 2 hours ago | parent | prev [-] | ||||||||||||||||
I mean not believing intelligence score is reasonable but cost is just reporting what they spent? | |||||||||||||||||
| |||||||||||||||||