Conformal prediction
A method that surrounds any model's prediction with an interval or set carrying a coverage guarantee, for example 90 percent, without distributional assumptions, to tell when a model does not know.
Most learning models give a point prediction, an amount or a probability, without saying how reliable it is for the case at hand. Conformal prediction, developed from the work of Vladimir Vovk and his coauthors in the early 2000s, builds around each prediction an interval, or for a classification a set of possible classes, that contains the true value with a probability chosen in advance, for example 90 percent. Its strength is to work with any model and without assumptions about the shape of the data distribution, relying on the model's errors measured on a separate calibration sample. The guarantee assumes only that new cases statistically resemble those used for calibration. In insurance, the method meets a concrete need: distinguishing cases where the model is confident from those where it is not. A very wide interval on a claim's cost signals a file for an adjuster; a narrow interval allows automatic handling. It does not protect against a change in the population, which breaks the resemblance assumption, and must therefore be recalibrated when the data drift.
Textbook case: a model estimates the final cost of bodily injury claims at notification. With 90 percent conformal prediction, one file gets an estimate of 25,000 euros with an interval of 18,000 to 34,000 euros and goes to standard handling; another, also estimated at 25,000 euros but with an interval of 5,000 to 400,000 euros, is assigned straight away to a specialist handler.
Follow the research
One letter a week. First, the week's underwriting note, when one is published: a dated, argued decision, signed by Paul Bernardski. Then what the observatory published on cyber risk, political risk and insurance. No account, no tracking.
No spam, one-click unsubscribe, no data ever sold.