Remix.run Logo
mort96 13 hours ago

As it seems to be getting worse over time (with 4.8 being worse than 4.6 and 5 worse than 4.8 again), could this simply be signs of model collapse?

I mean more and more training data you find on the web is generated by previous models. The only reliable way to find human-generated text is to find text written before 2022, and they've used up all of that already. And AFAIU, these companies are using more and more synthetic data or semi-synthetic data.