Why would I adapt myself to unseen environments unnecessarily?

You can see possible existing environments even if you explicitly blind yourself to them, through reflections off of environments that you do not blind yourself to. Even with the benchmark excluded, the social zeitgeist that has considered the benchmark and included it or ideas from the benchmark either implicitly or explicitly in their code, documentation, et cetera is still part of your training data.

Because then you could sit under the palm tree and enjoy your free bananas and relax.

Propagation