False positives (fraud detection)

On January 15, 2021, Mark Rutte's Dutch government resigned over the toeslagenaffaire: for years, the tax administration had treated thousands of parents receiving childcare benefits as fraudsters, on the strength of flags some of which came from a risk classification model. A false positive, in fraud detection, is an honest claim that the system flags as suspicious. Its cost is twofold. The honest policyholder sees settlement delayed, faces questions that treat him as a suspect and, often, leaves at renewal. The special investigations unit spends its hours on files with no fraud, at the expense of those that contain some. The problem starts with rarity. Suppose 1% of claims are fraudulent and a model catches 90% of frauds while wrongly flagging only 5% of honest files: out of 10,000 claims, it raises 90 correct alerts and 495 false ones, so that barely more than 15% of alerts point to fraud. This is the base rate fallacy, and it explains why a model that looks excellent on paper swamps the investigators. Tuning happens through the threshold. Raising it cuts false positives and lets more fraud through, lowering it does the opposite. Practitioners therefore reason in terms of investigation capacity rather than accuracy: they set the number of alerts the team can work each week and measure precision on that slice, using a lift curve rather than an overall AUC. The law pulls the same way, since a denial based solely on automated processing falls under article 22 of the GDPR. In practice, an alert opens a check and does not on its own justify a denial, but the line blurs quickly when the handler rubber-stamps alerts out of habit.