I don't think it can give you story beats you shouldn't be aware of yet. Those beats wouldn't be fed into the prompt until the event happens. LLM can't spit out what it doesn't know.

It might indeed fail to reveal something it should but even that i think is unlikely if the harness steers it hard enough.

I think it could be fun. If you're always given 4 choices of what you can ask the NPC then your choices can be too obvious. If its open ended then you have to think a little what to say and ask.

Why couldn’t an LLM accidentally spit out story beats if they plausibly follow from the context and the player’s input into the conversation?