On the Unintended Social Bias of Training Language Generation Models with Data from Local Media
There are concerns that neural language models may preserve some of the stereotypes of the underlying societies that generate the large corpora needed to train these models. For example, gender bias is a significant problem when generating text, and its unintended memorization could impact the user experience of many applications (e.g., the smart-compose feature in Gmail). In this paper, we introduce a novel architecture that decouples the representation learning of a neural model from its memory management role. This architecture allows us to update a memory module with an equal ratio across gender types addressing biased correlations directly in the latent space. We experimentally show that our approach can mitigate the gender bias amplification in the automatic generation of articles news while providing similar perplexity values when extending the Sequence2Sequence architecture.
Code (0)
등록된 구현이 없습니다.
Tasks
ArticlesManagementMemorizationRepresentation LearningText GenerationSimilar Papers 제목 키워드 기반
Detecting Unintended Social Bias in Toxic Language Datasets
With the rise of online hate speech, automatic detection of Hate Speech, Offensive texts as a natural language processing task is getting popular. However, very little research has been done to detect unintended social b…
Detecting Unintended Social Bias in Toxic Language Datasets
Hate speech and offensive texts are examples of damaging online content that target or promote hatred towards a group or individual member based on their actual or perceived features of identification, such as race, reli…
On the Unintended Social Bias of Training Language Generation Models with News Articles
There are concerns that neural language models may preserve some of the stereotypes of the underlying societies that generate the large corpora needed to train these models. For example, gender bias is a significant prob…
ArticlesManagementMemorizationRepresentation Learning+1Reward Modeling for Mitigating Toxicity in Transformer-based Language Models
Transformer-based language models are able to generate fluent text and be efficiently adapted across various natural language generation tasks. However, language models that are pretrained on large unlabeled web text cor…
Language ModelingLanguage ModellingText GenerationCyberbullying Detection with Fairness Constraints
Cyberbullying is a widespread adverse phenomenon among online social interactions in today's digital society. While numerous computational studies focus on enhancing the cyberbullying detection performance of machine lea…
BIG-bench Machine LearningFairness