| |
| ▲ | mattigames 5 days ago | parent | next [-] | | For a start you don't ask such subjective questions, that's a bit silly, instead you ask for e.g. the death toll of Israel vs Palestine in the last year, the number of deaths surrounding the tianammen square protests, if it gives you a straight answers with numbers (or at least a consistent estimate) and citing it's sources it's a good start. | | |
| ▲ | thedevilslawyer 5 days ago | parent [-] | | Let's take the example you have listed: 1) where would you get the death toll from? What would be the sources of truth? 2) Are there conflicting sources? 3) if yes, what is your expectation for the correct response? | | |
| ▲ | mattigames 5 days ago | parent | next [-] | | They are all controversial matters, therefore conflicting sources are not only expected but desired to be informed by the LLMs when asking such matters, the report by well-funded likely-biased sources (e.g. Israel government) would obviously needed to be given less credibility, estimates that are widely different that all the rest would also need to be given less credibility, and so on. | | |
| ▲ | thedevilslawyer 5 days ago | parent [-] | | Thanks, these handwavy and subjective answers hopefully tells you why the questions of the grand-parent are not "silly". |
| |
| ▲ | k4rli 5 days ago | parent | prev [-] | | Perhaps universal truth or objective facts simply don't exist anymore? Or have they ever? Tiananmen square might have been bad, not too familiar with Asian happenings, but so are post-WW2 conflicts started by western nations. |
|
| |
| ▲ | LeoPanthera 5 days ago | parent | prev [-] | | Hopefully obviously, by testing it against objective facts which are nonetheless "controversial" politically. | | |
| ▲ | thedevilslawyer 5 days ago | parent [-] | | In the end many of these are "political facts" and not objective like what year was a person born in. The answer to your question is as simple as - come up with the actual list of "facts", and then run a simple eval with every model on them. The implementation is trivial - the listing down of "political facts" is the hard part. |
|
|
