Leveraging Per-Instance Privacy for Machine Unlearning
We present a principled, per-instance approach to quantifying the difficulty of unlearning via fine-tuning. We begin by sharpening an analysis of noisy gradient descent for unlearning (Chien et al., 2024), obtaining a better utility-unlearning tradeoff by replacing worst-case privacy loss bounds with per-instance privacy losses (Thudi et al., 2024), each of which bounds the (Renyi) divergence to retraining without an individual data point. To demonstrate the practical applicability of our theory, we present empirical results showing that our theoretical predictions are born out both for Stochastic Gradient Langevin Dynamics (SGLD) as well as for standard fine-tuning without explicit noise. We further demonstrate that per-instance privacy losses correlate well with several existing data difficulty metrics, while also identifying harder groups of data points, and introduce novel evaluation methods based on loss barriers. All together, our findings provide a foundation for more efficient and adaptive unlearning strategies tailored to the unique properties of individual data points.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine UnlearningSimilar Papers 제목 키워드 기반
Efficient Unlearning with Privacy Guarantees
Privacy protection laws, such as the GDPR, grant individuals the right to request the forgetting of their personal data not only from databases but also from machine learning (ML) models trained on them. Machine unlearni…
SecureCut: Federated Gradient Boosting Decision Trees with Efficient Machine Unlearning
In response to legislation mandating companies to honor the \textit{right to be forgotten} by erasing user data, it has become imperative to enable data removal in Vertical Federated Learning (VFL) where multiple parties…
Federated LearningMachine UnlearningVertical Federated LearningBenchmarking Federated Machine Unlearning methods for Tabular Data
Machine unlearning, which enables a model to forget specific data upon request, is increasingly relevant in the era of privacy-centric machine learning, particularly within federated learning (FL) environments. This pape…
BenchmarkingComputational EfficiencyFederated LearningMachine Unlearning+1Faster Machine Unlearning via Natural Gradient Descent
We address the challenge of efficiently and reliably deleting data from machine learning models trained using Empirical Risk Minimization (ERM), a process known as machine unlearning. To avoid retraining models from scra…
Computational EfficiencyMachine UnlearningTowards Efficient Target-Level Machine Unlearning Based on Essential Graph
Machine unlearning is an emerging technology that has come to attract widespread attention. A number of factors, including regulations and laws, privacy, and usability concerns, have resulted in this need to allow a trai…
Machine Unlearning