>weighted flat earth and spherical earth

It should default to the mainstream opinion in general, but if you specifically ask about the shape of Earth, I do think it should mention the different theories as well as their pluses, minuses, and consensus. Avoiding bad or outdated ideas is bad in both theory and practice. If we don't speak about it, then that leaves the microphone solely to the kooks. I want an AI that gives me all relevant positions. Otherwise, it's a waste of my time. I don't want to go to my AI priest for quick answers. I want to be informed.

Most things won't fall onto such easily decidable lines as well. In politics, which encompass far more than people realize, there will be many opinions that are not falsifiable. As an example, if someone asks, "What country is Kashmire in?" Would it be more informative to give one side of the story or both? Which one is "correct?"

>trained to ignore structured argument

AI is not rational (yet), although some would argue that it's getting close. It is complex retrieval of highly curated, unstructured data. So yes, there is enormous bias in the information, some of it good, some of it bad.

This supports multiple positions. AIs do not think. They do not reason. Their best use is summarizing subjects while giving multiple citations that I can read myself. The AI user motto must be, "Trust but verify."