| ▲ | babelfish 6 hours ago |
| Taking the benchmarks in the article at face value, this is ~on par with Opus 5.5 for ~1/4 the price! |
|
| ▲ | mwigdahl 6 hours ago | parent | next [-] |
| Which is why I don't take the benchmarks at face value... |
| |
| ▲ | babelfish 6 hours ago | parent [-] | | So you have decided the model is bad without having used it yet? | | |
| ▲ | cbg0 6 hours ago | parent | next [-] | | If they had a model on par with Opus 5.5 there would be no need to undercut it by that much, that's probably the logic. | | |
| ▲ | rfgplk 5 hours ago | parent [-] | | Anthropic models are comically overpriced. They also gobble up an obscene # of tokens to get the same result as others. The price delta is literally 5x. |
| |
| ▲ | mwigdahl 4 hours ago | parent | prev [-] | | No, I have decided I don't trust benchmarks without using the model first. |
|
|
|
| ▲ | entrope 5 hours ago | parent | prev | next [-] |
| Taking the benchmarks (as reported by OpenAI) at face value, why wouldn't they show Opus 5.5 on the OS world benchmark, and show no Anthropic models at all on the "factual error rate" benchmark |
|
| ▲ | enraged_camel 6 hours ago | parent | prev [-] |
| ...no it's not? At max effort, it scores 50% on terminal bench vs. Opus 5.5's 70%. |
| |
| ▲ | babelfish 6 hours ago | parent [-] | | ALE (top benchmark in post) is 60% for Terra vs 63% for Opus - I guess we'll have to wait for AA Index! | | |
|
