| ▲ | WarmWash 3 hours ago | |
Maybe this is a neither can confirm or deny thing, but are there systems in place or design decisions made that are meant to surface attempts at benchmark optimizing (benchmaxxing), outside of just having private sets? Something like a heuristic anti-cheat I suppose. Or perhaps the view is that any gains are good gains? Like studying for a test by leaning on brute memorization is still a non-zero positive gain. | ||
| ▲ | fchollet 2 hours ago | parent [-] | |
There are no tricks. Our approach to reducing the impact of targeting (without fully eliminating it) is described in the paper. | ||