| ▲ | CitrusFruits an hour ago | |
I could be wrong, but I think Artificial Analysis typically runs benchmarks independently after models become publicly available. | ||
| ▲ | lambda 4 minutes ago | parent [-] | |
A lot of times they are given early access so they can release benchmarks on the day of the model public release. But the odd thing is that even the previous generations of these models aren't on Artificial Analysis. It's been months since version 1 of some of these models was announced, but none of these are on AA. Are they only providing them with anti-benchmarking language or something? | ||