Model Collapse

You cannot see model collapse happening from the outside.

It does not arrive like a blackout. The screen does not go dark. The system does not suddenly forget how to speak. There is no siren, no rupture, no dramatic moment when the machine confesses that something essential has been lost.

It still writes in complete sentences. It still summarizes documents. It still produces code that looks plausible. It still explains concepts with the clean confidence of a consultant in a pressed shirt. It still passes many of the tests placed in front of it. It may even improve on some of them.

“The system does not suddenly forget how to speak.”

But beneath the surface, the distribution has begun to narrow.

The difficult edge conditions lose resolution.

The model becomes smoother, safer, more generic, and more confident at the exact same time. It does not become obviously stupid. It becomes statistically average in a way that can masquerade as intelligence until the moment reality demands something more than average.

To understand why this happens, you have to strip away the mythology and look at what a language model is actually doing.