> impervious to groupthink, cliques, and organised campaigns

Yeeeeah, no. LLMs are only as good as the datasets they are trained on (ie the internet, with all its "personality"). We also know the output is highly influenced by the prompting, which is a human-determined parameter, and this seems unlikely to change any time soon.

This idea that the potential of AI/LLMs is somehow not fairly represented by how they're currently used is ludicrous to me. There is no utopia in which their behaviour is somehow magically separated from the source of their datasets. While society continues to elevate and amplify the likes of Musk, the AI will simply reflect this, and no version of LLM-pedia will be a truly viable alternative to Wikipedia.

The core problem is that AI training processes can't by itself know during training that a part of the training dataset is bad.

Basically, a normal human with some basic media literacy knows that tabloids, the "yellow press" rags, Infowars or Grokipedia aren't good authoritative sources and automatically downranks their content or refuses to read it entirely.

An AI training program however? It can't skip over B.S., it relies on the humans compiling the dataset - otherwise it will just ingest it and treat it as 1:1 ranked with authoritative, legitimate sources.