Reconstructing a proprietary model by repeatedly querying its interface, transferring value without crossing any security perimeter.
A model extraction attack consists in querying a prediction interface a very large number of times with chosen inputs, then training a local model on the resulting input and response pairs until a substitute of comparable performance is obtained. It requires no intrusion, no stolen credentials and no software vulnerability, since it uses only the service as sold, which makes it invisible to security devices and hard to characterize legally. The harm is nonetheless real and twofold: the value of the training investment is transferred, and the substitute then serves to prepare further attacks, adversarial examples built against it often transferring to the original model. The usual countermeasures are all trade-offs between protection and service quality, comprising query rate limiting, reduced precision in returned scores, detection of systematic querying patterns and model watermarking. For insurance this exposure belongs to intellectual property rather than to a conventional cyber policy, a distinction many wordings do not yet make.
The paper Stealing Machine Learning Models via Prediction APIs, presented at USENIX Security in 2016, showed that a commercial model exposed through a prediction interface could be reconstructed with high fidelity from a limited number of queries.
model extraction, vol de modèle, model stealing, attaque par extraction