Data labeling
The work of assigning each example the correct answer, fraud or not, damaged part, clause present, on which a supervised model will learn, whose quality sets the ceiling on the model's performance.
A supervised model learns from examples whose answer is known: the label. In some cases the label follows naturally from the data, like the final cost of a closed claim. In many others it must be produced: deciding whether a file was fraudulent, marking the damaged parts on a photo, stating whether a policy contains a given exclusion, classifying the cause of a claim. This annotation work, often entrusted to business experts, is costly and slow, and its quality determines the model's performance ceiling: a model cannot be more accurate than the labels it learns from. Three difficulties are specific to insurance. Labels are often incomplete: only detected frauds are known, not the others, so the model learns to reproduce investigators' criteria rather than to recognize fraud. They are sometimes inconsistent, since two experts may classify the same file differently. And they age as practices change. Hence the importance of written annotation protocols, measuring agreement between annotators and regularly reviewing labeled sets.
Textbook case: to train a motor damage valuation model, an insurer has its adjusters annotate 20,000 photos. A check shows that two adjusters agree on whether to repair or replace a part in only 78 percent of cases; the insurer writes a common protocol, re-annotates the disputed cases, and agreement rises to 91 percent before training.
Follow the research
One letter a week. First, the week's underwriting note, when one is published: a dated, argued decision, signed by Paul Bernardski. Then what the observatory published on cyber risk, political risk and insurance. No account, no tracking.
No spam, one-click unsubscribe, no data ever sold.