Bridging the Fairness Gap: Enhancing Pre-trained Models with LLM-Generated Sentences
Pre-trained language models (PLMs) are trained on data that inherently contains gender biases, leading to undesirable impacts. Traditional debiasing methods often rely on external corpora, which may lack quality, diversity, or demographic balance, affecting the effectiveness of debiasing. With the rise of large language models and their extensive knowledge, we propose enhancing fairness (Fair-Gender) in PLMs by absorbing coherent, attribute-balanced, and semantically rich sentences. However, these sentences cannot be directly used for debiasing due to alignment issues and the risk of negative transfer. We address this by applying causal analysis to estimate causal effects, filtering out unaligned sentences, and identifying aligned ones for incorporation into PLMs, thereby ensuring positive transfer. Experiments show that our approach significantly reduces gender biases in PLMs while preserving their language expressiveness.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeDiversityFairnessSimilar Papers 제목 키워드 기반
Human-Guided Fair Classification for Natural Language Processing
Text classifiers have promising applications in high-stake tasks such as resume screening and content moderation. These classifiers must be fair and avoid discriminatory decisions by being invariant to perturbations of s…
ClassificationFairnessStyle TransferToward Fairness in Text Generation via Mutual Information Minimization based on Importance Sampling
Pretrained language models (PLMs), such as GPT2, have achieved remarkable empirical performance in text generation tasks. However, pretrained on large-scale natural language corpora, the generated text from PLMs may exhi…
FairnessLanguage ModelingLanguage ModellingSentence+1Widespread Gender and Pronoun Bias in Moral Judgments Across LLMs
Large language models (LLMs) are increasingly used to assess moral or ethical statements, yet their judgments may reflect social and linguistic biases. This work presents a controlled, sentence-level study of how grammat…
SAFES: Sequential Privacy and Fairness Enhancing Data Synthesis for Responsible AI
As data-driven and AI-based decision making gains widespread adoption in most disciplines, it is crucial that both data privacy and decision fairness are appropriately addressed. While differential privacy (DP) provides …
Decision MakingFairnessBridging Dictionary: AI-Generated Dictionary of Partisan Language Use
Words often carry different meanings for people from diverse backgrounds. Today's era of social polarization demands that we choose words carefully to prevent miscommunication, especially in political communication and j…
Language ModelingLanguage ModellingLarge Language Model