Remix.run Logo
▲ ranger_danger 2 hours ago

Which others?

▲benaduggan 2 hours ago | parent [-]

There are several public benchmarks for decision models where they don't top every single one. https://benchmarkheaven.com/jev-models/all

I guess their cheeky "Warning: AI Benchmarks, don't trust them" is their way out of that one.

▲woadwarrior01 17 minutes ago | parent [-]

The way that site charges a fee[1] for benchmarking models, makes any benchmarks on it very sus, IMO. TBF, the benchmarking site is probably more profitable than TypeSafe, just like the joke about how WinRAR is more profitable than OpenAI[2]. :)

[1]: https://benchmarkheaven.com/submit

[2]: https://news.ycombinator.com/item?id=49797811