Nonparametric Variational Differential Privacy via Embedding Parameter Clipping
The nonparametric variational information bottleneck (NVIB) provides the foundation for nonparametric variational differential privacy (NVDP), a framework for building privacy-preserving language models. However, the learned latent representations can drift into regions with high information content, leading to poor privacy guarantees, but also low utility due to numerical instability during training. In this work, we introduce a principled parameter clipping strategy to directly address this issue. Our method is mathematically derived from the objective of minimizing the Rényi Divergence (RD) upper bound, yielding specific, theoretically grounded constraints on the posterior mean, variance, and mixture weight parameters. We apply our technique to an NVIB based model and empirically compare it against an unconstrained baseline. Our findings demonstrate that the clipped model consistently achieves tighter RD bounds, implying stronger privacy, while simultaneously attaining higher performance on several downstream tasks. This work presents a simple yet effective method for improving the privacy-utility trade-off in variational models, making them more robust and practical.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Differential Privacy for Transformer Embeddings of Text with Nonparametric Variational Information Bottleneck
We propose a privacy-preserving method for sharing text data by sharing noisy versions of their transformer embeddings. It has been shown that hidden representations learned by deep models can encode sensitive informatio…
Embedding-Based Federated Data Sharing via Differentially Private Conditional VAEs
Deep Learning (DL) has revolutionized medical imaging, yet its adoption is constrained by data scarcity and privacy regulations, limiting access to diverse datasets. Federated Learning (FL) enables decentralized training…
Federated LearningAssessing Differentially Private Variational Autoencoders under Membership Inference
We present an approach to quantify and compare the privacy-accuracy trade-off for differentially private Variational Autoencoders. Our work complements previous work in two aspects. First, we evaluate the the strong reco…
Time SeriesTime Series AnalysisMinimax And Adaptive Transfer Learning for Nonparametric Classification under Distributed Differential Privacy Constraints
This paper considers minimax and adaptive transfer learning for nonparametric classification under the posterior drift model with distributed differential privacy constraints. Our study is conducted within a heterogeneou…
ClassificationTransfer LearningDifferential Privacy in Personalized Pricing with Nonparametric Demand Models
In the recent decades, the advance of information technology and abundant personal data facilitate the application of algorithmic personalized pricing. However, this leads to the growing concern of potential violation of…
Adversarial Attack