It's cute that you think your high-school level cypher is probably not seen in the training set of one of the biggest LLMs in the world. Surely no one could have thought of such a cypher, let alone create exercises around it!

No one should ever make claims such as "X is not in <LLM>'s training set". You don't know. Even if your idea is indeed original, nothing prevents someone from having though of it before, and published it. The history of science is full of simultaneous discoveries, and we're talking cutting-edge research.

The point is not that the cypher is hard, the point is that the randomish string it needs to answer the question can’t possibly be computed just from correlations from the training data. Rather, it learned an emergent, generalizable skill that it used to solve it.