Does Differential Privacy Impact Bias in Pretrained NLP Models?
Differential privacy (DP) is applied when fine-tuning pre-trained large language models (LLMs) to limit leakage of training examples. While most DP research has focused on improving a model's privacy-utility tradeoff, some find that DP can be unfair to or biased against underrepresented groups. In this work, we show the impact of DP on bias in LLMs through empirical analysis. Differentially private training can increase the model bias against protected groups w.r.t AUC-based bias metrics. DP makes it more difficult for the model to differentiate between the positive and negative examples from the protected groups and other groups in the rest of the population. Our results also show that the impact of DP on bias is not only affected by the privacy protection level but also the underlying distribution of the dataset.
Code (1)
Similar Papers 제목 키워드 기반
How Does Differential Privacy Affect Social Bias in LLMs? A Systematic Evaluation
Large language models (LLMs) trained on web-scale corpora can memorize sensitive training data, posing significant privacy risks. Differential privacy (DP) has emerged as a principled framework that limits the influence …
Question AnsweringRemoving Disparate Impact of Differentially Private Stochastic Gradient Descent on Model Accuracy
When we enforce differential privacy in machine learning, the utility-privacy trade-off is different w.r.t. each group. Gradient clipping and random noise addition disproportionately affect underrepresented and complex c…
The Impact of Differential Privacy on Group Disparity Mitigation
The performance cost of differential privacy has, for some applications, been shown to be higher for minority groups; fairness, conversely, has been shown to disproportionally compromise the privacy of members of such gr…
FairnessThe Impact of Differential Privacy on Recommendation Accuracy and Popularity Bias
Collaborative filtering-based recommender systems leverage vast amounts of behavioral user data, which poses severe privacy risks. Thus, often, random noise is added to the data to ensure Differential Privacy (DP). Howev…
Collaborative FilteringRecommendation SystemsPosition: Considerations for Differentially Private Learning with Large-Scale Public Pretraining
The performance of differentially private machine learning can be boosted significantly by leveraging the transfer learning capabilities of non-private models pretrained on large public datasets. We critically review thi…
PositionPrivacy PreservingTransfer Learning