It would be an interesting feedback loop to capture a screenshot of a game (sm64), determine the green texture is grass, update the texture realtime, take another screenshot, and iterate, having the game flesh itself out realtime as it’s being played.

Four years ago there was demo of exactly that in GTA5 https://www.ign.com/articles/experimental-ai-tool-makes-gta-...