Wouldn't you get crazy overfitting?
Not if you look for a minimal diff across a range of tasks.
[dead]