| ▲ | paxys 3 hours ago | |
How is every company able to show itself at the top of every benchmark? | ||
| ▲ | morgengold 2 hours ago | parent | next [-] | |
First look what models are worse in a set of self selected benchmarks. Second, compare to older versions of competitor s models. Still does not look good? Compare to own previous models. | ||
| ▲ | adam_arthur 3 hours ago | parent | prev | next [-] | |
Not much moat, incremental improvements, cherry picking models to compare. To be fair, seems more correct to compare against similar strength models if your main edge is pricing. | ||
| ▲ | toephu2 an hour ago | parent | prev | next [-] | |
Anyone deep in the AI realm know which is the gold standard benchmark for coding? | ||
| ▲ | ffsm8 2 hours ago | parent | prev | next [-] | |
They're being called "trust me bro benchmarks" for a reason ( ・ั ﹏ ・ั ) | ||
| ▲ | logicchains 2 hours ago | parent | prev [-] | |
Wait to the exact moment your model is ahead on at least N benchmarks then publish. | ||