Remix.run Logo
▲ tintor an hour ago

Why are these new models not on Artificial Analysis?

▲CitrusFruits an hour ago | parent [-]

I could be wrong, but I think Artificial Analysis typically runs benchmarks independently after models become publicly available.

▲lambda a minute ago | parent [-]

A lot of times they are given early access so they can release benchmarks on the day of the model public release.

But the odd thing is that even the previous generations of these models aren't on Artificial Analysis. It's been months since version 1 of some of these models was announced, but none of these are on AA. Are they only providing them with anti-benchmarking language or something?