Inferring patterns in unfamiliar problems.

Take a common word problem in a 5th grade math text book. Now, change as many words as possible; instead of two trains, make it two different animals; change the location to a rarely discussed town; etc. Even better, invent words/names to identify things.

Someone who has done a word problem like that will very likely recognize the logic, even if the setting is completely different.

Word tokenization alone should fail miserably.

I have noted over my life that a lot of problems end up being a variation on solved problems from another more familiar domain but frustratingly take a long time to solve before realizing this was just like that thing you had already solved. Nevertheless, I do feel like humans do benefit from identifying meta patterns but as the chess example shows even we might be weak in unfamiliar areas.

A lot of LLMs do weird things on the question "A farmer needs to get a bag of grain across a river. He has a boat that can transport himself and the grain. How does he do this?"

(they often pattern-match on the farmer/grain/sheep/fox puzzle and start inventing pointless trips ("the farmer returns alone. Then, he crosses again.") in a way that a human wouldn't)