You're putting a lot of trust into the judgement abilities of what is just a next token predictor there.

I can see what the goals are there, and they do make sense I suppose, but I'm not confident that what you're handing off there can be handed off to that degree.

But maybe that is not the point and the point instead is to see what the LLM thinks would be correct, and then think about that and collect learnings about the world from it. It might not be right, but it still tells you how normal people think. So that's useful.

Just a very roundabout way to achieve that, but that's fine, I guess.