Benchmaxxing is the default case, and always has been.

It's really, really difficult to avoid it even when you care to stop yourself; and it's not even just a problem in machine learning, it's the standard failure mode of all minds capable of learning, human, animal, artificial.

Even pure genetics has this problem. Viruses and cancers also demonstrate this behaviour, with the bench being evolution's only option: reproductive success.

▲

conmod278 an hour ago | parent [-]

Why would I adapt myself to unseen environments unnecessarily?

	▲	NewJazz 24 minutes ago \| parent \| next [-]
		You can see possible existing environments even if you explicitly blind yourself to them, through reflections off of environments that you do not blind yourself to. Even with the benchmark excluded, the social zeitgeist that has considered the benchmark and included it or ideas from the benchmark either implicitly or explicitly in their code, documentation, et cetera is still part of your training data.
	▲	Muromec 36 minutes ago \| parent \| prev \| next [-]
		Because then you could sit under the palm tree and enjoy your free bananas and relax.
	▲	TedDoesntTalk 41 minutes ago \| parent \| prev [-]
		Propagation