De-amplifying Bias from Differential Privacy in Language Model Fine-tuning
Fairness and privacy are two important values machine learning (ML) practitioners often seek to operationalize in models. Fairness aims to reduce model bias for social/demographic sub-groups. Privacy via differential privacy (DP) mechanisms, on the other hand, limits the impact of any individual's training data on the resulting model. The trade-offs between privacy and fairness goals of trustworthy ML pose a challenge to those wishing to address both. We show that DP amplifies gender, racial, and religious bias when fine-tuning large language models (LLMs), producing models more biased than ones fine-tuned without DP. We find the cause of the amplification to be a disparity in convergence of gradients across sub-groups. Through the case of binary gender bias, we demonstrate that Counterfactual Data Augmentation (CDA), a known method for addressing bias, also mitigates bias amplification by DP. As a consequence, DP and CDA together can be used to fine-tune models while maintaining both fairness and privacy.
Code (0)
등록된 구현이 없습니다.
Tasks
counterfactualData AugmentationFairnessLanguage ModelingLanguage ModellingSimilar Papers 제목 키워드 기반
Amplifying Rényi Differential Privacy via Shuffling
Differential privacy is a useful tool to build machine learning models which do not release too much information about the training data. We study the R\'enyi differential privacy of stochastic gradient descent when each…
BIG-bench Machine LearningDoes Differential Privacy Impact Bias in Pretrained NLP Models?
Differential privacy (DP) is applied when fine-tuning pre-trained large language models (LLMs) to limit leakage of training examples. While most DP research has focused on improving a model's privacy-utility tradeoff, so…
Resource-Adaptive Federated Text Generation with Differential Privacy
In cross-silo federated learning (FL), sensitive text datasets remain confined to local organizations due to privacy regulations, making repeated training for each downstream task both communication-intensive and privacy…
Federated LearningText GenerationDifferentially Private Bias-Term Fine-tuning of Foundation Models
We study the problem of differentially private (DP) fine-tuning of large pre-trained models -- a recent privacy-preserving approach suitable for solving downstream tasks with sensitive data. Existing work has demonstrate…
Privacy PreservingTechnical Report: Full Version of Analyzing and Optimizing Perturbation of DP-SGD Geometrically
Differential privacy (DP) has become a prevalent privacy model in a wide range of machine learning tasks, especially after the debut of DP-SGD. However, DP-SGD, which directly perturbs gradients in the training iteration…