▲ | simianwords 8 days ago | |
I think its a good idea but how do you not accidentally benchmark hack here? | ||
▲ | GabrielBianconi 7 days ago | parent [-] | |
We set up dataset splits and the usual best practices. Of course, if you overdo things, you can still hack benchmarks; our goal isn't to publish SOTA numbers but rather to illustrate results from our methodology. We didn't even tune hyperparameters, we just used the default choices. Definitely a valid concern for teams chasing SOTA though. Thanks! |