Noise-Aware Differentially Private Variational Inference
Differential privacy (DP) provides robust privacy guarantees for statistical inference, but this can lead to unreliable results and biases in downstream applications. While several noise-aware approaches have been proposed which integrate DP perturbation into the inference, they are limited to specific types of simple probabilistic models. In this work, we propose a novel method for noise-aware approximate Bayesian inference based on stochastic gradient variational inference which can also be applied to high-dimensional and non-conjugate models. We also propose a more accurate evaluation method for noise-aware posteriors. Empirically, our inference method has similar performance to existing methods in the domain where they are applicable. Outside this domain, we obtain accurate coverages on high-dimensional Bayesian linear regression and well-calibrated predictive probabilities on Bayesian logistic regression with the UCI Adult dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceregressionVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DP-IVON-Gradsq: Differentially Private Squared-Gradient Improved Variational Online Newton
Differential privacy provides formal privacy guarantees for training neural networks on sensitive data, while Bayesian deep learning offers a principled framework for uncertainty-aware prediction. Combining these two obj…
Computational EfficiencyDPVIm: Differentially Private Variational Inference Improved
Differentially private (DP) release of multidimensional statistics typically considers an aggregate sensitivity, e.g. the vector norm of a high-dimensional vector. However, different dimensions of that vector might have …
Variational InferenceAn end-to-end Differentially Private Latent Dirichlet Allocation Using a Spectral Algorithm
We provide an end-to-end differentially private spectral algorithm for learning LDA, based on matrix/tensor decompositions, and establish theoretical guarantees on utility/consistency of the estimated model parameters. T…
SensitivityVariational InferenceDifferentially Private Variational Inference for Non-conjugate Models
Many machine learning applications are based on data collected from people, such as their tastes and behaviour as well as biological traits and genetic data. Regardless of how important the application might be, one has …
Bayesian InferenceVariational InferencePrivate Posterior distributions from Variational approximations
Privacy preserving mechanisms such as differential privacy inject additional randomness in the form of noise in the data, beyond the sampling mechanism. Ignoring this additional noise can lead to inaccurate and invalid i…
Privacy Preservingvalid