| ▲ | danielmarkbruce 2 hours ago |
| 100% this. Lots of people want cheaper. The demand for same performance but cheaper is off the charts. |
|
| ▲ | danudey an hour ago | parent | next [-] |
| Agree. I was talking to a coworker the other day and remarked that I would be pretty content if we had five years of Claude stagnating in quality/capability but reducing in price/environmental/electrical impact. It reminds me of when Apple switched to their own silicon in the iPhones and we had year after year of hardware with 1.5-2.5x performance compared to the previous year. We very rapidly got to the point where the processor in my phone was more power than I would ever need (unless apps got more and more bloated and inefficient, which they obviously did). |
|
| ▲ | Ampersander 2 hours ago | parent | prev | next [-] |
| The performance is really fuzzy too, but the price you do know for certain. |
| |
| ▲ | mcintyre1994 2 hours ago | parent [-] | | Price is quite fuzzy too though when it's pricing tokens, because a model that's more token efficient but more expensive per token can be cheaper for a given task. And token efficiency is as fuzzy as performance so you're back where you started. | | |
| ▲ | seizethecheese an hour ago | parent [-] | | > a model that's more token efficient but more expensive per token can be cheaper for a given task. True, but I'm skeptical that this is now overly used to ignore cost per token. Your comment inspired me to do a quick investigation. Looks like in my data, 92% of cost per task can be predicted by cost per token https://micjm.com/blog/cost-per-task-vs-token/ | | |
| ▲ | danielmarkbruce an hour ago | parent [-] | | Definitely. Cost per task is nice in theory, pretty absurd in practice. | | |
| ▲ | seizethecheese 40 minutes ago | parent [-] | | Well, it depends on the data source. Cost per task for benchmarks is absurd. Cost per task for real world usage, where the model identity is randomized (Like Arena or Agent Duel) seems better, though the tasks still are not going to be representative if you're not doing the same stuff as the battle participants. |
|
|
|
|
|
| ▲ | santoshalper 2 hours ago | parent | prev [-] |
| Or even 90% of the performance but much cheaper. Many business tasks to not require frontier intelligence and were solvable two years ago. |
| |
| ▲ | adventured 2 hours ago | parent [-] | | Yeah if you're talking 85-90% of the performance at 10-15% the cost, you're going to win most of the business use cases over. That has been the broad theoretical appeal of the Chinese models, that they might provide that kind of value. Most businesses could not care less about open weights vs not, that does not enter into their discussions very heavily. They care about cost to performance. Fable 5.2 is amazing, and it's as good at eating dollars as it is at producing code. |
|