Indirect discrimination
Lawful factors correlated with a protected variable reproduce its effect, and machine learning makes the detour undetectable.
In its Test-Achats judgment of March 1, 2011 (C-236/09), the Court of Justice of the European Union struck down the exemption in Article 5(2) of Directive 2004/113/EC. From December 21, 2012, premiums and benefits could no longer vary by sex. The ban covers the variable; its effect remains. Neutral-looking variables (occupation, vehicle type, place of residence, browsing history) stand in for a protected characteristic as soon as they correlate with it, and that is indirect discrimination. A model trained on hundreds of correlated variables reconstitutes the characteristic unbidden, often better than a direct declaration would, since several imperfect proxies are worth one reliable indicator. The ban mainly takes away the means to measure the effect, because the data needed to audit the model is no longer collected. The algorithmic fairness literature calls this strategy fairness through unawareness and generally rates it the weakest of all. It is nonetheless, in substance, the one European law adopted. The proposed remedies (collecting the protected variable for audit purposes only, constraining the training, correcting gaps after the fact) run into a symmetric legal difficulty: measuring requires collecting, and correcting requires using.
A study of 2011 to 2013 data established that direct gender discrimination, once banned, reconstituted itself through occupation and vehicle type.
Follow the research
One letter a week. First, the week's underwriting note, when one is published: a dated, argued decision, signed by Paul Bernardski. Then what the observatory published on cyber risk, political risk and insurance. No account, no tracking.
No spam, one-click unsubscribe, no data ever sold.