Data poisoning
An attack that deliberately corrupts a model's training data to bias its behavior or to insert a backdoor.
In 2017 Tianyu Gu, Brendan Dolan-Gavitt and Siddharth Garg published BadNets: a network trained on tampered data reads stop signs correctly until someone sticks a yellow Post-it on one, and then it reads a speed limit. Poisoning goes after what the model learns from, its training data, rather than the trained model itself. The attacker slips corrupted examples into the training set. That can degrade the model across the board or, more quietly, hide a backdoor that fires only on a specific trigger the attacker alone knows. It is hard to catch, because the model behaves normally almost all the time. And it feeds on huge corpora, often scraped from the internet, whose provenance and integrity nobody can fully vouch for. The attack surface moves upstream in the AI production chain: this is supply chain risk, applied to models. For insurers the loss is latent and deferred. The damage is planted long before it shows, which makes it very hard to date the triggering event and tie it to a policy period.
Textbook case: an attacker inserts, into the public data used to train a model, booby-trapped examples that teach it to misclassify a specific type of document. The flaw remains invisible until the attacker, months later, exploits it against the deployed model.
Follow the research
One letter a week. First, the week's underwriting note, when one is published: a dated, argued decision, signed by Paul Bernardski. Then what the observatory published on cyber risk, political risk and insurance. No account, no tracking.
No spam, one-click unsubscribe, no data ever sold.