There is/was an AI Dungeon app running in the early/pre-ChatGPT days using GPT 3-ish, I think? Long term context was a real problem - story arcs were very.... drifty. A more modern agentic approach might help with this, doing multiple passes over the work to achieve consistency.

Nah, it wouldn't work. LLMs don't have coherent, total concepts of things in the same way humans do.

AIDungeon was built on a fine-tuned version of GPT-2. Less coherent but infinitely more fun than later models.