Back to glossaryAI

Adversarial examples

Inputs specially crafted to fool a machine-learning model into producing an incorrect output, while being indistinguishable from normal inputs to a human.

Definition

Adversarial examples are input data to which small, often imperceptible, perturbations have been deliberately added to induce a classification or prediction error in a machine-learning model. The phenomenon was demonstrated in 2014 by Goodfellow et al. on vision neural networks: a correctly classified panda image can be pixel-perturbed to be classified as a gibbon with high confidence, without the perturbation being visible to the naked eye. Adversarial robustness is now a central security criterion for any model deployed in adversarial environments: facial recognition systems, fraud detection, autonomous driving, insurance document processing. White-box attacks, where the attacker knows the model architecture and weights, are distinguished from black-box attacks, where only outputs are available. In the LLM context, adversarial examples take the form of carefully crafted prompts designed to bypass guardrails. For an insurer using a credit scoring or fraud detection model, vulnerability to adversarial examples constitutes a model risk to be integrated into the ORSA and algorithmic governance.

Example

An auto insurer deploys a vision model to assess damage from claims photographs. A dishonest repair expert submits photos slightly altered by a digital filter that artificially reduces the damage estimate by 40%. The model is not robust to adversarial perturbations, leading to systematic underestimation of claims discovered several months later.

Related terms
Also known as

adversarial examples, exemples adversariaux, perturbations adversariales, adversarial inputs, exemples adverses