Two things:

1. This is a 2022 era picture that LLMs just suck up what's on the Internet and regurgitate it. It hasn't been true for a long time.

2. There's almost no one who has a stronger incentive to distinguish LLM generated text from non-generated LLM text. Training classifiers is pretty manageable.

This is not true.

The other day for me Google search AI cited triggerbites which is a vibe data website.

The LLM unless explicitly prompted cannot distinguish the validity of a data source.

The article literally describes ChatGPT regurgitating something it sucked up on the Internet to this day.

>1. This is a 2022 era picture that LLMs just suck up what's on the Internet and regurgitate it. It hasn't been true for a long time.

When I ask ChatGPT certain questions, it very clearly shows me the sources it's searching through in the form of favicons, and they're always the favicons of sites that are in the first 5-10 results for a Google search query that would fit what I've asked the LLM. It will then cite those websites as it's source, with a direct link to the page that also shows up in an equivalent Google search. It does that daily.

>1. This is a 2022 era picture that LLMs just suck up what's on the Internet and regurgitate it. It hasn't been true for a long time

if anything it’s more true now, AI chatbots rarely use raw LLM output, instead they have CoT and use search tool calls. and the speed at which webpages are crawled is in the matter of hours. I posted something here on hn, then searched for it on google 1 hr later and its AI answered with my comment

https://share.google/aimode/PkQUVfA68AlHGb7Yr

there’s several companies right now making a killing of organic placement on chatbots similar to the SEO rush of early google

> 2) There's almost no one who has a stronger incentive to distinguish LLM generated text from non-generated LLM text. Training classifiers is pretty manageable

i think it’s pretty naive to think that if LLM companies would develop LLM detection they would use it to skip training on that content.

instead, they use it to train their output to be less detectable.

LLM detection is not solvable because it is adversarial, an arms race

I'm not sure you get my point. Just ask ChatGPT, "What are the top 10 xxx," and see which sources it cites. Oftentimes, it's complete garbage with LLM tells.

> 2. There's almost no one who has a stronger incentive to distinguish LLM generated text from non-generated LLM text.

There's a tension there: they want you to use AI for everything, and if they get what they want then new information will be embedded in LLM text.

> Training classifiers is pretty manageable.

Hopefully such things can be opened up, so the slop can be iterated on until it passes. It doesn't need to be perfect, just enough so more garbage gets through.

[dead]