Membership Inference Attacks Cannot Prove that a Model Was Trained On Your Data
We consider the problem of a training data proof, where a data creator or owner wants to demonstrate to a third party that some machine learning model was trained on their data. Training data proofs play a key role in recent lawsuits against foundation models trained on web-scale data. Many prior works suggest to instantiate training data proofs using membership inference attacks. We argue that this approach is fundamentally unsound: to provide convincing evidence, the data creator needs to demonstrate that their attack has a low false positive rate, i.e., that the attack's output is unlikely under the null hypothesis that the model was not trained on the target data. Yet, sampling from this null hypothesis is impossible, as we do not know the exact contents of the training set, nor can we (efficiently) retrain a large foundation model. We conclude by offering two paths forward, by showing that data extraction attacks and membership inference on special canary data can be used to create sound training data proofs.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
On the (In)Feasibility of Attribute Inference Attacks on Machine Learning Models
With an increase in low-cost machine learning APIs, advanced machine learning models may be trained on private datasets and monetized by providing them as a service. However, privacy researchers have demonstrated that th…
AttributeBIG-bench Machine LearningInference AttackDo Backdoors Assist Membership Inference Attacks?
When an adversary provides poison samples to a machine learning model, privacy leakage, such as membership inference attacks that infer whether a sample was included in the training of the model, becomes effective by mov…
Inference AttackMembership Inference AttackMembership Inference Attacks for Unseen Classes
Shadow model attacks are the state-of-the-art approach for membership inference attacks on machine learning models. However, these attacks typically assume an adversary has access to a background (nonmember) data distrib…
quantile regressionregressionOn the Discredibility of Membership Inference Attacks
With the wide-spread application of machine learning models, it has become critical to study the potential data leakage of models trained on sensitive data. Recently, various membership inference (MI) attacks are propose…
Reconstruction and Membership Inference Attacks against Generative Models
We present two information leakage attacks that outperform previous work on membership inference against generative models. The first attack allows membership inference without assumptions on the type of the generative m…
Density EstimationInference AttackMembership Inference Attack