Gini coefficient and AUC (discriminatory power)
Measures of a model's ability to rank risks from best to worst, independently of the level of its predictions, used to compare tariffs or fraud scores.
A pricing or scoring model can be wrong in two ways: on level, by predicting too much or too little overall, and on ranking, by putting good and bad risks in the wrong order. The Gini coefficient and the area under the ROC curve measure the second quality, discriminatory power. For a binary classification, fraud or not, claim or not, the ROC curve plots the true positive rate against the false positive rate for every possible threshold, and its area, the AUC, equals 0.5 for a model ranking at random and 1 for perfect ranking. The Gini coefficient is linked to it by a simple relation, equal to twice the area minus one. In pricing, a version adapted to amounts is used, based on the concentration curve of losses ordered by predicted premium. These measures serve to compare two models on the same data, for example a linear model and a gradient boosting model. They say nothing about calibration: a model can rank perfectly and predict wrong amounts, so they must always be complemented by a calibration measure and business judgment.
Textbook case: a fraud score reaches an area under the curve of 0.82 on the test sample, a Gini of 0.64, against 0.74 and 0.48 for the expert rules it replaces. By investigating the top-scored 5 percent of files, the insurer finds twice as many confirmed frauds as with the old rules.
Follow the research
One letter a week. First, the week's underwriting note, when one is published: a dated, argued decision, signed by Paul Bernardski. Then what the observatory published on cyber risk, political risk and insurance. No account, no tracking.
No spam, one-click unsubscribe, no data ever sold.