| ▲ | asdfologist an hour ago | |
Meh, I doubt it was intentional. Deliberate benchmaxxing is incredibly damaging to credibility once it's discovered (see what happened to Meta with LlaMa 4). It's more likely that the training data was contaminated with the benchmark data. | ||
| ▲ | mupuff1234 an hour ago | parent [-] | |
You really think they saw the jump in arc-agi-3 (which they reported in their official card), and didn't even bother to check? They maybe have not intentionally benchmaxxed, but they certainly know that's what happened . | ||