Privacy-preserving Learning via Deep Net Pruning
This paper attempts to answer the question whether neural network pruning can be used as a tool to achieve differential privacy without losing much data utility. As a first step towards understanding the relationship between neural network pruning and differential privacy, this paper proves that pruning a given layer of the neural network is equivalent to adding a certain amount of differentially private noise to its hidden-layer activations. The paper also presents experimental results to show the practical implications of the theoretical finding and the key parameter values in a simple practical setting. These results show that neural network pruning can be a more effective alternative to adding differentially private noise for neural networks.
Code (0)
등록된 구현이 없습니다.
Tasks
Network PruningPrivacy PreservingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Privacy-Preserving-Oriented DNN Pruning and Mobile Acceleration Framework
Weight pruning of deep neural networks (DNNs) has been proposed to satisfy the limited storage and computing capability of mobile edge devices. However, previous pruning methods mainly focus on reducing the model size an…
Model CompressionPrivacy PreservingArtemis: HE-Aware Training for Efficient Privacy-Preserving Machine Learning
Privacy-Preserving ML (PPML) based on Homomorphic Encryption (HE) is a promising foundational privacy technology. Making it more practical requires lowering its computational cost, especially, in handling modern large de…
Model CompressionPrivacy PreservingCutClean: Neural Network Pruning for Privacy-Preserving Inference
Neural networks are increasingly deployed in high-stakes applications with growing privacy leakage concerns. We show that this privacy leakage can occur even in the absence of representation imbalances that lead to tradi…
Network PruningFedSpaLLM: Federated Pruning of Large Language Models
Large Language Models (LLMs) achieve state-of-the-art performance but are challenging to deploy due to their high computational and storage demands. Pruning can reduce model size, yet existing methods assume public acces…
Federated LearningPrivacy PreservingExploring Federated Pruning for Large Language Models
LLM pruning has emerged as a promising technology for compressing LLMs, enabling their deployment on resource-limited devices. However, current methodologies typically require access to public calibration samples, which …
Privacy Preserving