Attack against a machine-learning model that reconstructs representative inputs from training data using the model's outputs, enabling the recovery of private or confidential information.
The model inversion attack was formalized by Fredrikson et al. in 2015 in a paper showing that by repeatedly querying a pharmaceutical prediction model on fictitious patients, it was possible to reconstruct facial images of patients used to train a medical facial recognition model. It differs from a membership inference attack in that it does not seek to determine whether a specific individual was in the training set, but to reconstruct generic representations of the private data that shaped the model. Modern model inversion techniques on image generators, known as GAN-inversion, enable reconstruction of high-resolution images of faces or medical documents by optimizing an input to produce the target output. For insurers operating pricing models trained on confidential medical or behavioral data, this type of attack raises GDPR compliance questions, notably regarding the data minimization principle and Article 25 on privacy by design. The main defense is differential privacy during training, which perturbs gradients to limit memorization, at the cost of a slight degradation in predictive performance. This trade-off between accuracy and privacy protection has become a central arbitrage in AI governance in insurance.
A security researcher queries the API of an insurer offering instant health quotes based on a learning model. By submitting 50,000 synthetic profiles and analyzing score variations, they reconstruct typical medical profiles that maximize the estimated risk. These reconstructed profiles correspond precisely to elderly diabetic patients from the training set, whose data had been pseudonymized but not rendered private in the differential sense.
model inversion, inversion de modèle, reconstruction de données d'entraînement, model inversion attack