But wouldn't (couldn't) the model then hallucinate play patterns and get itself into problems when playing against a real opponent?

Well, if your training includes regularization against ruleset changes, the model should simply handle it. (that would be the expensive option, and require vastly more training)

When the Dota 2 bot was made, they retrained the bot only partially when new patches came in, so it was definitely cheaper to adapt.