>But it felt as if they are using this to "avoid" some of the hard questions, and we dropped this bucket to force them to provide a verdict.

do you not see how that creates extremely misleading and valueless results? you are coercing the results into what you want to see.

Exactly what people do when they use LLMs for "fact-checking" online, and any verbose explanation would be mostly ignored anyway, when people ask political, ethical, or simply ambiguous questions that they hold any stakes in.

Don't even need politics for it, there is no point in probing a mathematical black box for "how many soldiers died in the year X in war Y".

Any original source is preferable to a blurry "summary" of unknown sources, and this is why the article has a valuable point.

There's also no point in asking "Is Paris in France" either, if you substitute city and country with real data. An encyclopedia or manual check of different sources such as maps, while not infallible, is a better source.

If you already know the country Paris belongs to, there's no point in asking, anyway.