Model collapse
The progressive degradation of a generative model trained, generation after generation, on content produced by other models: diversity and rare cases vanish from its outputs.
Large language and image models were trained on content produced by humans. As AI-generated content multiplies online, later models risk being trained partly on the outputs of earlier ones. Research published in 2024 in the journal Nature showed the resulting mechanism: each generation over-represents the most frequent cases and forgets the tails of the distribution, so that over iterations the model produces increasingly uniform, then degraded, outputs. Rare cases are the first to vanish. The phenomenon is not inevitable: it can be limited by keeping original human data, tracing the provenance of training data and controlling the share of synthetic data. It matters to insurance in two ways. First for insurers using synthetic data to train pricing or fraud detection models: a model that loses the tail of the distribution loses precisely what counts in insurance, rare and severe claims. Second for assessing the risk of companies deploying AI: a model's quality is not a given, it depends on the data chain feeding it, and can degrade without any visible signal.
Textbook case: an insurer enriches its claims data with synthetic claims generated by a model, then retrains that model on the augmented set three years in a row. On review, the simulated frequency of claims above one million has been divided by four, while reality has not changed: this is the phenomenon described in Nature in July 2024.
Follow the research
One letter a week. First, the week's underwriting note, when one is published: a dated, argued decision, signed by Paul Bernardski. Then what the observatory published on cyber risk, political risk and insurance. No account, no tracking.
No spam, one-click unsubscribe, no data ever sold.