Skip to content
←Back to glossaryAI

Adversarial examples

Inputs specially crafted to fool a machine-learning model into producing an incorrect output, while being indistinguishable from normal inputs to a human.

Definition

Adversarial examples are inputs to which small, often imperceptible perturbations have been deliberately added to push a machine-learning model into a classification or prediction error. Szegedy et al. demonstrated the phenomenon on vision neural networks ("Intriguing properties of neural networks", ICLR 2014); Goodfellow et al. explained it by the linearity of the models in late 2014, with the fast gradient sign method (FGSM) and the image that became famous. A correctly classified panda image, perturbed at the pixel level, gets classified as a gibbon with high confidence, and the change is invisible to the naked eye. Resistance to adversarial inputs is now a central security criterion for any model deployed in hostile environments: facial recognition, fraud detection, autonomous driving, insurance document processing. In a white-box attack, the attacker knows the model's architecture and weights; in a black-box attack, he sees only its outputs. With LLMs, adversarial examples take the form of carefully crafted prompts that bypass guardrails. For an insurer running a credit scoring or fraud detection model, this vulnerability is a model risk to fold into the ORSA and algorithmic governance.

Its 8 cross-references, woven across the glossary's 18 categories
Adversarial examples: AI glossary illustration, AlgoPolis
Example

Textbook case: an auto insurer deploys a vision model to assess damage from claims photographs. A dishonest repair expert submits photos slightly altered by a digital filter that artificially inflates the damage estimate by 40%. The model does not hold up under adversarial perturbations. Claims are systematically overpaid, and nobody notices for several months.

Read next
AIData poisoningAn attack that deliberately corrupts a model's training data to bias its behavior or to insert a backdoor.Read the entry →AIAlgorithmic black boxA system whose outputs are observable but whose internal path remains unexplainable, even to its designers.Read the entry →AIExplainability (explainable AI)The ability to make an AI model's decisions understandable, a matter of trust, regulatory compliance and liability.Read the entry →
Put it to work
QuizHow a model fails, and who answers for itAcademy questions on this concept, each one marked and explained.Test yourself →
Sources
  • Goodfellow, Shlens and Szegedy, "Explaining and Harnessing Adversarial Examples", arXiv 1412.6572, December 2014 (ICLR 2015), citing Szegedy et al., "Intriguing properties of neural networks", ICLR 2014
Related terms
Go further at the Academy
Also known as

adversarial examples, adversarial inputs, adversarial perturbations

Stay informed

Follow the research

One letter a week. First, the week's underwriting note, when one is published: a dated, argued decision, signed by Paul Bernardski. Then what the observatory published on cyber risk, political risk and insurance. No account, no tracking.

No spam, one-click unsubscribe, no data ever sold.