> In some sense it proves to me that llms are not just copying and are actually creating a bespoke solution.

Is the weirdness of the designs meaningfully beneficial in any way that the AI has figured out, or could it for example be that the training data the LLMs learned from are too different from the type of data it would need to learn from in order to interface properly and play the game better? And then because of that incompleteness, the LLM has to hallucinate a lot and it results in weird outcomes.

No their weird designs are not better in any way - yet.