Distributed Differentially Private Computation of Functions with Correlated Noise
Many applications of machine learning, such as human health research, involve processing private or sensitive information. Privacy concerns may impose significant hurdles to collaboration in scenarios where there are multiple sites holding data and the goal is to estimate properties jointly across all datasets. Differentially private decentralized algorithms can provide strong privacy guarantees. However, the accuracy of the joint estimates may be poor when the datasets at each site are small. This paper proposes a new framework, Correlation Assisted Private Estimation (CAPE), for designing privacy-preserving decentralized algorithms with better accuracy guarantees in an honest-but-curious model. CAPE can be used in conjunction with the functional mechanism for statistical and machine learning optimization problems. A tighter characterization of the functional mechanism is provided that allows CAPE to achieve the same performance as a centralized algorithm in the decentralized setting using all datasets. Empirical results on regression and neural network problems for both synthetic and real datasets show that differentially private methods can be competitive with non-private algorithms in many scenarios of interest.
Code (2)
Tasks
BIG-bench Machine LearningPrivacy PreservingSimilar Papers 제목 키워드 기반
Distributed Differentially-Private Algorithms for Matrix and Tensor Factorization
In many signal processing and machine learning applications, datasets containing private information are held at different locations, requiring the development of distributed privacy-preserving algorithms. Tensor and mat…
Privacy PreservingTensor DecompositionCorrelated Privacy Mechanisms for Differentially Private Distributed Mean Estimation
Differentially private distributed mean estimation (DP-DME) is a fundamental building block in privacy-preserving federated learning, where a central server estimates the mean of $d$-dimensional vectors held by $n$ users…
Federated LearningPrivacy PreservingDP-ADMM: ADMM-based Distributed Learning with Differential Privacy
Alternating Direction Method of Multipliers (ADMM) is a widely used tool for machine learning in distributed settings, where a machine learning model is trained over distributed data sources through an interactive proces…
BIG-bench Machine LearningLocally Differentially Private Online Federated Learning With Correlated Noise
We introduce a locally differentially private (LDP) algorithm for online federated learning that employs temporally correlated noise to improve utility while preserving privacy. To address challenges posed by the correla…
Federated LearningDifferentially Private Decentralized Dataset Synthesis Through Randomized Mixing with Correlated Noise
In this work, we explore differentially private synthetic data generation in a decentralized-data setting by building on the recently proposed Differentially Private Class-Centric Data Aggregation (DP-CDA). DP-CDA synthe…
Synthetic Data Generation