Model calibration
The quality of a model whose predictions match, in level, the frequencies and amounts actually observed, essential in insurance where the premium must be right in value, not just well ranked.
A model is well calibrated when its predictions match outcomes in level: among files to which it assigns a 10 percent fraud probability, about one in ten should actually be fraudulent; among policyholders for whom it predicts an average cost of 300 euros, the observed cost should be close to 300 euros. Calibration is distinct from discriminatory power: a model can rank risks perfectly while systematically overestimating their cost, and vice versa. In insurance it is essential, because the premium is an amount that must cover losses: a well-ranked but poorly calibrated tariff produces losses or lost market share. Several modern learning methods, trained to rank, produce poorly calibrated scores, which must be recalibrated through a monotone transformation fitted on a separate sample. Calibration is checked with charts comparing predictions and outcomes by band, and with statistical tests. It degrades over time, with claims inflation and data drift, which requires monitoring it after deployment and not just at validation.
A frequency model trained on 2020 to 2023 predicts an average frequency of 6.2 percent in 2025, while observed frequency reaches 7.1 percent, due to rising climate claims. Its ranking remains good, but it systematically underestimates the level: the team recalibrates it before renewing the tariff.
Follow the research
One letter a week. First, the week's underwriting note, when one is published: a dated, argued decision, signed by Paul Bernardski. Then what the observatory published on cyber risk, political risk and insurance. No account, no tracking.
No spam, one-click unsubscribe, no data ever sold.