Transfer Learning In Differential Privacy's Hybrid-Model

2022-01-28 · Refael Kohen, Or Sheffet

The hybrid-model (Avent et al 2017) in Differential Privacy is a an augmentation of the local-model where in addition to N local-agents we are assisted by one special agent who is in fact a curator holding the sensitive details of n additional individuals. Here we study the problem of machine learning in the hybrid-model where the n individuals in the curators dataset are drawn from a different distribution than the one of the general population (the local-agents). We give a general scheme -- Subsample-Test-Reweigh -- for this transfer learning problem, which reduces any curator-model DP-learner to a hybrid-model learner in this setting using iterative subsampling and reweighing of the n examples held by the curator based on a smooth variation of the Multiplicative-Weights algorithm (introduced by Bun et al, 2020). Our scheme has a sample complexity which relies on the chi-squared divergence between the two distributions. We give worst-case analysis bounds on the sample complexity required for our private reduction. Aiming to reduce said sample complexity, we give two specific instances our sample complexity can be drastically reduced (one instance is analyzed mathematically, while the other - empirically) and pose several directions for follow-up work.

📄 PDF Abstract BibTeX arXiv:2201.12018

Code (0)

등록된 구현이 없습니다.

Tasks

modelTransfer Learning

NoEsis: Differentially Private Knowledge Transfer in Modular LLM Adaptation

2025-04-25 · Rob Romijnders, Stefanos Laskaridis, Ali Shahin Shamsabadi, Hamed Haddadi

Large Language Models (LLM) are typically trained on vast amounts of data from various sources. Even when designed modularly (e.g., Mixture-of-Experts), LLMs can leak privacy on their sources. Conversely, training such m…

Code Completion Mixture-of-Experts parameter-efficient fine-tuning Transfer Learning

Cross-Network Social User Embedding with Hybrid Differential Privacy Guarantees

2022-09-04 · Jiaqian Ren, Lei Jiang, Hao Peng, Lingjuan Lyu 외

Integrating multiple online social networks (OSNs) has important implications for many downstream social mining tasks, such as user preference modelling, recommendation, and link prediction. However, it is unfortunately …

Attribute Link Prediction Network Embedding Privacy Preserving

BLENDER: Enabling Local Search with a Hybrid Differential Privacy Model

2017-05-02 · Brendan Avent, Aleksandra Korolova, David Zeber, Torgeir Hovden 외

We propose a hybrid model of differential privacy that considers a combination of regular and opt-in users who desire the differential privacy guarantees of the local privacy model and the trusted curator model, respecti…

The power of synergy in differential privacy: Combining a small curator with local randomizers

2019-12-18 · Amos Beimel, Aleksandra Korolova, Kobbi Nissim, Or Sheffet 외

Motivated by the desire to bridge the utility gap between local and trusted curator models of differential privacy for practical applications, we initiate the theoretical study of a hybrid model introduced by "Blender" […

Two-sample testing

HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning

2019-12-12 · Runhua Xu, Nathalie Baracaldo, Yi Zhou, Ali Anwar 외

Federated learning has emerged as a promising approach for collaborative and privacy-preserving learning. Participants in a federated learning process cooperatively train a model by exchanging model parameters instead of…

Federated Learning Privacy Preserving