The Privacy Onion Effect: Memorization is Relative
Machine learning models trained on private datasets have been shown to leak their private data. While recent work has found that the average data point is rarely leaked, the outlier samples are frequently subject to memorization and, consequently, privacy leakage. We demonstrate and analyse an Onion Effect of memorization: removing the "layer" of outlier points that are most vulnerable to a privacy attack exposes a new layer of previously-safe points to the same attack. We perform several experiments to study this effect, and understand why it occurs. The existence of this effect has various consequences. For example, it suggests that proposals to defend against memorization without training with rigorous privacy guarantees are unlikely to be effective. Further, it suggests that privacy-enhancing technologies such as machine unlearning could actually harm the privacy of other users.
Code (1)
Tasks
Machine UnlearningMemorizationSimilar Papers 제목 키워드 기반
Towards Differential Relational Privacy and its use in Question Answering
Memorization of the relation between entities in a dataset can lead to privacy issues when using a trained model for question answering. We introduce Relational Memorization (RM) to understand, quantify and control this …
MemorizationQuestion AnsweringSpores: Stateless Predictive Onion Routing for E-Squads
Mass surveillance of the population by state agencies and corporate parties is now a well-known fact. Journalists and whistle-blowers still lack means to circumvent global spying for the sake of their investigations. Wit…
On the Privacy Effect of Data Enhancement via the Lens of Memorization
Machine learning poses severe privacy concerns as it has been shown that the learned models can reveal sensitive information about their training data. Many works have investigated the effect of widely adopted data augme…
Adversarial RobustnessData AugmentationMemorizationBatch Normalization Amplifies Memorization and Privacy Risks
Batch Normalization (BN) is widely adopted to enable faster convergence and more stable training of deep neural networks. However, its impact on privacy and memorization has remained largely unexplored. In this work, we …
Investigating Data Memorization in 3D Latent Diffusion Models for Medical Image Synthesis
Generative latent diffusion models have been established as state-of-the-art in data generation. One promising application is generation of realistic synthetic medical imaging data for open data sharing without compromis…
Contrastive LearningImage GenerationMemorization