Remix.run Logo
paxys 3 hours ago

How is every company able to show itself at the top of every benchmark?

morgengold 2 hours ago | parent | next [-]

First look what models are worse in a set of self selected benchmarks.

Second, compare to older versions of competitor s models.

Still does not look good? Compare to own previous models.

adam_arthur 3 hours ago | parent | prev | next [-]

Not much moat, incremental improvements, cherry picking models to compare.

To be fair, seems more correct to compare against similar strength models if your main edge is pricing.

toephu2 an hour ago | parent | prev | next [-]

Anyone deep in the AI realm know which is the gold standard benchmark for coding?

ffsm8 2 hours ago | parent | prev | next [-]

They're being called "trust me bro benchmarks" for a reason ( ・ั ﹏ ・ั )

logicchains 2 hours ago | parent | prev [-]

Wait to the exact moment your model is ahead on at least N benchmarks then publish.