Yeah, the internet seems like a big poison pill. Training on the whole internet feels like citing the National Enquirer (or the Daily Mail?) for a school essay.

Having an archive of "curated" training data seems like it is going to be important. Otherwise you need "AS" (artificial skepticism) introduced into future models. ("But I read it on the internet!", ha ha.)

Or perhaps there are ways to bucket training data such that the model is aware of which data leans factual (quantifiable) and which data leans opinion (fuzzy, qualifiable?).

(I recently asked Claude about the existence of ball lightning, spontaneous human combustion. I got replies that ultimately did not leave me satisfied. It's probably just as well that I read this article though—I now have an even stronger degree of skepticism with regard to their replies—specifically, I suppose, with topics that are likely to be biased.)

(I'm not quite convinced from the article though that Google is "fighting back". In fact, this feels like another moment where a "player" could try to establish their LLM as more factual. Is that the row Grok is trying to hoe? Or is Grok just trying to be anti-woke?)

> Having an archive of "curated" training data seems like it is going to be important

the justification for not doing that is probably "prohibitively expensive given the amount of data involved". they'd need a bunch of human reviewers combing through massive troves of data. it's probably cheaper to "sort of fix" it after the fact.

> perhaps there's ways to bucket training data such that the model is aware of which data leans factual (quantifiable) and which data leans opinion (fuzzy, qualifiable)

as a lecturer once said to me about my idea for a masters dissertation project that would classify news sites based on right/left tendencies -- "that sounds dangerously political". especially given the current let's all shout at each other political climate.

aside: someone built this and it was a fully fledged company, which has always annoyed me.

> Training on the whole internet feels like citing the National Enquirer

It's not, though, because the refutations are in the training data too. This isn't actually the problem being described.

The weights in the LLM are fine. It's that the task the LLM is being asked to do is to search and summarize new content that isn't in its training data. And it does it too much like a naive reader and not enough like a cynical HN commenter.

But that's a problem with prompt writing, not training. It's also of a piece with most of the other complaints about current AI solutions, really: AI still lacks the "context" that an experienced human is going to apply, so it doesn't know when it's supposed to reason and when it's supposed to repeat.

If you were to ask it "Is this site correct or is it just spin?" it will probably get it right. But it doesn't know to ask itself that question if it's not in the prompt somewhere.