Towards Debiasing Sentence Representations
As natural language processing methods are increasingly deployed in real-world scenarios such as healthcare, legal systems, and social science, it becomes necessary to recognize the role they potentially play in shaping social biases and stereotypes. Previous work has revealed the presence of social biases in widely used word embeddings involving gender, race, religion, and other social constructs. While some methods were proposed to debias these word-level embeddings, there is a need to perform debiasing at the sentence-level given the recent shift towards new contextualized sentence representations such as ELMo and BERT. In this paper, we investigate the presence of social biases in sentence-level representations and propose a new method, Sent-Debias, to reduce these biases. We show that Sent-Debias is effective in removing biases, and at the same time, preserves performance on sentence-level downstream tasks such as sentiment analysis, linguistic acceptability, and natural language understanding. We hope that our work will inspire future research on characterizing and removing social biases from widely adopted sentence representations for fairer NLP.
Code (1)
Tasks
Linguistic AcceptabilityNatural Language UnderstandingSentenceSentiment AnalysisWord EmbeddingsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoders
Pretrained text encoders, such as BERT, have been applied increasingly in various natural language processing (NLP) tasks, and have recently demonstrated significant performance gains. However, recent studies have demons…
Contrastive LearningFairnessSentenceA Simple yet Effective Self-Debiasing Framework for Transformer Models
Current Transformer-based natural language understanding (NLU) models heavily rely on dataset biases, while failing to handle real-world out-of-distribution (OOD) instances. Many methods have been proposed to deal with t…
Natural Language UnderstandingSentenceTowards Debiasing Temporal Sentence Grounding in Video
The temporal sentence grounding in video (TSGV) task is to locate a temporal moment from an untrimmed video, to match a language query, i.e., a sentence. Without considering bias in moment annotations (e.g., start and en…
SentenceTemporal Sentence GroundingProjective Methods for Mitigating Gender Bias in Pre-trained Language Models
Mitigation of gender bias in NLP has a long history tied to debiasing static word embeddings. More recently, attention has shifted to debiasing pre-trained language models. We study to what extent the simplest projective…
Language ModelingLanguage ModellingSentenceWord EmbeddingsDebiasing Pre-trained Contextualised Embeddings
In comparison to the numerous debiasing methods proposed for the static non-contextualised word embeddings, the discriminative biases in contextualised embeddings have received relatively little attention. We propose a f…
SentenceWord Embeddings