Deep Learning with Data Privacy via Residual Perturbation
Protecting data privacy in deep learning (DL) is of crucial importance. Several celebrated privacy notions have been established and used for privacy-preserving DL. However, many existing mechanisms achieve privacy at the cost of significant utility degradation and computational overhead. In this paper, we propose a stochastic differential equation-based residual perturbation for privacy-preserving DL, which injects Gaussian noise into each residual mapping of ResNets. Theoretically, we prove that residual perturbation guarantees differential privacy (DP) and reduces the generalization gap of DL. Empirically, we show that residual perturbation is computationally efficient and outperforms the state-of-the-art differentially private stochastic gradient descent (DPSGD) in utility maintenance without sacrificing membership privacy.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningPrivacy PreservingSimilar Papers 제목 키워드 기반
Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning
The privacy leakage of the model about the training data can be bounded in the differential privacy mechanism. However, for meaningful privacy parameters, a differentially private model degrades the utility drastically w…
On Residual Networks Learning a Perturbation from Identity
The purpose of this work is to test and study the hypothesis that residual networks are learning a perturbation from identity. Residual networks are enormously important deep learning models, with many theories attemptin…
The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples
Machine unlearning offers a practical alternative to avoid full model re-training by approximately removing the influence of specific user data. While existing methods certify unlearning via statistical indistinguishabil…
DP-UTIL: Comprehensive Utility Analysis of Differential Privacy in Machine Learning
Differential Privacy (DP) has emerged as a rigorous formalism to reason about quantifiable privacy leakage. In machine learning (ML), DP has been employed to limit inference/disclosure of training examples. Prior work le…
BIG-bench Machine LearningInference AttackMembership Inference AttackregressionEdge Private Graph Neural Networks with Singular Value Perturbation
Graph neural networks (GNNs) play a key role in learning representations from graph-structured data and are demonstrated to be useful in many applications. However, the GNN training pipeline has been shown to be vulnerab…
Privacy Preserving