> I asked it “do you think the model will generate a meaningful number there?”

This also isn't likely to give you good results, models don't have a good sense of their own capabilities.

This is also in the same camp of asking a model why it did something. It'll generate something plausible and likely totally incorrect.

> models don't have a good sense of their own capabilities.

It actually had some memorized knowledge about LLMs being bad at confidence intervals, then it did a cursory web search. But it didn't need to do any of that; I already knew they would be bad numbers.