Skip to content
←Back to glossaryAI

Differential privacy

A mathematical guarantee that a statistical result barely changes whether or not any one person is in the data, obtained through calibrated noise.

Definition

The 2020 U.S. census: the Census Bureau protected its tables with differential privacy for the first time. It dropped the record swapping used in 2010 after finding that published tables were enough to reconstruct individual responses. Block-level figures lost accuracy as a result. Differential privacy describes a computation, not the data: it anonymizes nothing. It guarantees that the results have almost the same distribution whether or not a given individual appears in the dataset, and a publicly declared parameter bounds the maximum divergence. An observer therefore cannot conclude that the person is present, even holding all other information. That guarantee is far stronger than removing direct identifiers, which linkage has defeated many times. Its cost is explicit, and that is its virtue: the added noise degrades the accuracy of results, and the parameter makes public a trade-off a procedure would have hidden. Every query also draws down a budget, which forces uses to be counted. In machine learning, the property is obtained during training. It then bounds what a membership inference attack can learn, which makes it the only protection that can be demonstrated rather than merely observed.

Its 5 cross-references, woven across the glossary's 18 categories
Differential privacy: AI glossary illustration, AlgoPolis
Example

The United States Census Bureau applied differential privacy to the published results of the 2020 decennial census. It was the first implementation at that scale, and states challenged the decision in court over the lost precision in local data.

Read next
Related terms
Stay informed

Follow the research

One letter a week. First, the week's underwriting note, when one is published: a dated, argued decision, signed by Paul Bernardski. Then what the observatory published on cyber risk, political risk and insurance. No account, no tracking.

No spam, one-click unsubscribe, no data ever sold.