On Measuring and Mitigating Biased Inferences of Word Embeddings
Word embeddings carry stereotypical connotations from the text they are trained on, which can lead to invalid inferences in downstream models that rely on them. We use this observation to design a mechanism for measuring stereotypes using the task of natural language inference. We demonstrate a reduction in invalid inferences via bias mitigation strategies on static word embeddings (GloVe). Further, we show that for gender bias, these techniques extend to contextualized embeddings when applied selectively only to the static components of contextualized embeddings (ELMo, BERT).
Code (2)
Tasks
Natural Language InferenceWord EmbeddingsSimilar Papers 제목 키워드 기반
Detecting and Mitigating Indirect Stereotypes in Word Embeddings
Societal biases in the usage of words, including harmful stereotypes, are frequently learned by common word embedding methods. These biases manifest not only between a word and an explicit marker of its stereotype, but a…
AttributeWord EmbeddingsMitigating Political Bias in Language Models Through Reinforced Calibration
Current large-scale language models can be politically biased as a result of the data they are trained on, potentially causing serious problems when they are deployed in real-world settings. In this paper, we describe me…
reinforcement-learningReinforcement Learning (RL)Word EmbeddingsMeasuring Social Biases in Grounded Vision and Language Embeddings
We generalize the notion of social biases from language embeddings to grounded vision and language embeddings. Biases are present in grounded embeddings, and indeed seem to be equally or more significant than for ungroun…
Word EmbeddingsA Causal Inference Method for Reducing Gender Bias in Word Embedding Relations
Word embedding has become essential for natural language processing as it boosts empirical performances of various tasks. However, recent research discovers that gender bias is incorporated in neural word embeddings, and…
Causal Inferencecoreference-resolutionCoreference ResolutionSentence+1OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings
Language representations are known to carry stereotypical biases and, as a result, lead to biased predictions in downstream tasks. While existing methods are effective at mitigating biases by linear projection, such meth…
Word Embeddings