> Its made-up numbers have also ended up in an academic paper and in a bank's investment research.

It's an excellent way of showing how unreliable the information you find online is (and it will only get worse in the future), but willfully creating misinformation and made up shit and offering it as fact feels hostile to humanity.

Surely nobody got hurt as a consequence of those "facts", but bad decisions with real world impact could be made regardless. It feels like the behavior should be shunned in a general case (the same way I'd shun those numerous spam sites that pollute search results and very much would prefer they not be allowed to exist), even if in this case the intent was good.

Problem is, you can't really do anything about it, other than figuring out a way to take down hostile sites and keep them from popping up, so it's even worse than fraud or even just bad research in academia.

▲

miltonlost 3 hours ago | parent [-]

Part of the problem is also then the indiscriminate hoovering of any and all data by LLMs. If they kept their own trusted data sources instead of stealing from anyone and everything, then poisoned data wouldn't be an issue. The data poisoners are at fault, but so are the LLMs trusting anything and the companies thievery of human labor. Once the LLMs bring up this data, it's now on Anthropic and OpenAI for editorializing and saying "this is true".

Except for the fine print. LLMs are to be used for entertainment purposes. lol

▲

KronisLV 3 hours ago | parent [-]

> If they kept their own trusted data sources instead of stealing from anyone and everything, then poisoned data wouldn't be an issue.

If they didn't, they'd be unable to compete with their competition and also we'd get worse models that can't reason about many domains or do that much due to limited training data/generalization. I don't think we can put the cat back in the bag or mandate only non-problematic training codebases without shooting ourselves in the foot.

Now, if those AI orgs were mandated to properly compensate everyone whose data they've mined, to track which neurons get invoked and to pay accordingly based on what was used to train them... ehh, that's about as unrealistic as stopping. Kind of a bad situation all around, unless you abandon the idea of any sort of copyright.

	▲	miltonlost 3 hours ago \| parent [-]
		Sucks to be them but they deserve this for what they did to train in the first place. Nothing good enough to outweigh the bad they will produce will come from LLMs.