I've been calling this Software Collapse

It's the same problem that AI faces of Model Collapse: AIs that train on the internet ultimately just end up training on one another, stop moving forward, and end up as identical polished versions of one another

I now think of it as a Dr. Jekyll/ Mr. Hyde situation for software projects:

- Dr. Jekyll: For makers, the only limit is your imagination, architectural guidance, and token budget. Time to build!

- Mr. Hyde: For projects to get off the treadmill of having to copy others to maintain you position, you need to redefine how the project works and provides unique value. Features and quality are no longer the answer. Time to fight!

▲

kolinko 7 hours ago | parent [-]

That is not true - a model trained in the internet can both build verifiers to remove false/poor quality data from the next training, and build synthetic datasets that will supplement its training.

Similar to a human that wants to learn something and invents exercises to practice.

1-2 years ago it was a theory, but new models are trained, successfully, on synthetic datasets.

	▲	lmeyerov 7 hours ago \| parent [-]
		This feels like a bit of a semantic debate, but maybe a few useful perspectives, as interpret current bespoke work as not so rosy wrt collapse: - Synthetic datasets are typically human-steered today, which points to model collapse wrt learning from the internet. (Edit: or even simpler, model x data tapped out for cost/benefit even before collapse.) I don't think standard practice is (yet) AI looking at the internet and deciding to build its own gyms to go further. When it does, model collapse may happen again, and be even more expensive - Distillation attacks are getting interesting here. There seem to be 2 kinds: intentionally querying other models, and maybe not so intentionally, learning from reasoning traces going through shared routers, esp. coding ones - A lot of neolabs are trying to go where the big labs might not look as directly to avoid being squashed, which suggests they aren't ready to bet on being smarter, and that means the general AI is more about collapse / $