Evaluating Machine Unlearning via Epistemic Uncertainty
There has been a growing interest in Machine Unlearning recently, primarily due to legal requirements such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act. Thus, multiple approaches were presented to remove the influence of specific target data points from a trained model. However, when evaluating the success of unlearning, current approaches either use adversarial attacks or compare their results to the optimal solution, which usually incorporates retraining from scratch. We argue that both ways are insufficient in practice. In this work, we present an evaluation metric for Machine Unlearning algorithms based on epistemic uncertainty. This is the first definition of a general evaluation metric for Machine Unlearning to our best knowledge.
Code (1)
Tasks
Machine UnlearningSimilar Papers 제목 키워드 기반
Epistemic Deep Learning
The belief function approach to uncertainty quantification as proposed in the Demspter-Shafer theory of evidence is established upon the general mathematical models for set-valued observations, called random sets. Set-va…
Deep LearningUncertainty QuantificationEvaluating subgroup disparity using epistemic uncertainty in mammography
As machine learning (ML) continue to be integrated into healthcare systems that affect clinical decision making, new strategies will need to be incorporated in order to effectively detect and evaluate subgroup disparitie…
BIG-bench Machine LearningDecision MakingUncertainty QuantificationRedefining Machine Unlearning: A Conformal Prediction-Motivated Approach
Machine unlearning seeks to remove the influence of specified data from a trained model. While metrics such as unlearning accuracy (UA) and membership inference attack (MIA) provide baselines for assessing unlearning per…
Adversarial AttackConformal Predictionimage-classificationImage Classification+5Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Control
Uncertainty quantification is one of the central challenges for machine learning in real-world applications. In reinforcement learning, an agent confronts two kinds of uncertainty, called epistemic uncertainty and aleato…
continuous-controlContinuous Controlreinforcement-learningReinforcement Learning+2Evaluating deep learning models for fault diagnosis of a rotating machinery with epistemic and aleatoric uncertainty
Uncertainty-aware deep learning (DL) models recently gained attention in fault diagnosis as a way to promote the reliable detection of faults when out-of-distribution (OOD) data arise from unseen faults (epistemic uncert…
Fault Diagnosis