Supervised learning

In 2018, Alexander Noll, Robert Salzmann and Mario Wüthrich published a case study on a French motor third-party liability portfolio of about 678,000 policies, freely distributed as freMTPL2freq. They compare a generalized linear model, regression trees, gradient boosting and a neural network to predict claim counts. All of them are supervised learning. The term means training a model on examples whose answer, the label, is already known: for each past policy, its characteristics and the number of claims observed; for each claim file, whether it was fraudulent. The model learns the relationship between the variables and the target, then applies it to new cases. When the target is a number, such as frequency or average cost, it is called regression; when it is a category, classification. Actuaries have done this for a long time without using the word, since a GLM for claim frequency is a supervised model. Recent methods bring the ability to capture interactions the actuary had not specified, at the cost of explainability. In fraud, the method stumbles on its labels. Only detected fraud is known, and a file that was never investigated enters the data as honest. The model therefore learns what investigators already knew how to spot and reproduces their blind spots with flattering accuracy on the test set. Teams partly correct for this with randomly drawn audits, which give unbiased labels on a small sample, or by combining supervised learning with unsupervised learning, which looks for the unusual without labels. A high AUC measured on biased labels says nothing about the fraud nobody looked for.