Remix.run Logo
janalsncm 2 days ago

It isn’t pointless.

The author cited research that demonstrates that model collapse can happen on a small scale.

The author also cited sources that a larger and larger portion of the web will be written by language models.

There are already studies showing that LLM generated text is less diverse than human generated text:

https://techxplore.com/news/2026-03-llms-creativity-ai-respo...

https://arxiv.org/html/2501.19361

The studies don’t show that the lack of creativity in LLMs is caused by model collapse or that the problem is getting worse.

But 1) we know they do this and 2) we know that training on synthetic data can cause model collapse.

PeterisP 2 days ago | parent | next [-]

The key missing step which breaks the loop is that while indeed a larger and larger portion of the web is written by language models, that data isn't being used to train new models - at the beginning of LLMs people did indeed want to use "all the web" to train models, but that's not being done now anymore, you either take only old pre-LLM data, or you pay for new 'clean' data, or take extensive filtering steps to avoid accidentally ingesting synthetic data.

The main phrase of the title "model collapse is happening" is untrue and not substantiated in the article - all the true statements in the article are about the hypothetical problem, warning of the bad consequences that would likely happen if makers of major models did something they aren't doing, but they aren't doing that because that is a known issue that they're avoiding. It's like writing an article "Foot shooting epidemic is happening" with a long, solid (and true!) proof that if you'll shoot yourself in the foot, it will indeed cause serious injury...

simg a day ago | parent | prev [-]

>demonstrates that model collapse can happen

yes, so given the title one might expect cited research that model collapse IS happening, as per OP's point.