An Empirical Study on the Fairness of Pre-trained Word Embeddings
Pre-trained word embedding models are easily distributed and applied, as they alleviate users from the effort to train models themselves. With widely distributed models, it is important to ensure that they do not exhibit undesired behaviour, such as biases against population groups. For this purpose, we carry out an empirical study on evaluating the bias of 15 publicly available, pre-trained word embeddings model based on three training algorithms (GloVe, word2vec, and fastText) with regard to four bias metrics (WEAT, SEMBIAS,DIRECT BIAS, and ECT). The choice of word embedding models and bias metrics is motivated by a literature survey over 37 publications which quantified bias on pre-trained word embeddings. Our results indicate that fastText is the least biased model (in 8 out of 12 cases) and small vector lengths lead to a higher bias.
Code (0)
등록된 구현이 없습니다.
Tasks
FairnessWord EmbeddingsSimilar Papers 제목 키워드 기반
Impact of Gender Debiased Word Embeddings in Language Modeling
Gender, race and social biases have recently been detected as evident examples of unfairness in applications of Natural Language Processing. A key path towards fairness is to understand, analyse and interpret our data an…
FairnessLanguage ModelingLanguage ModellingWord EmbeddingsTowards Fair and Explainable AI using a Human-Centered AI Approach
The rise of machine learning (ML) is accompanied by several high-profile cases that have stressed the need for fairness, accountability, explainability and trust in ML systems. The existing literature has largely focused…
FairnessWord EmbeddingsAn Empirical Study of the Downstream Reliability of Pre-Trained Word Embeddings
While pre-trained word embeddings have been shown to improve the performance of downstream tasks, many questions remain regarding their reliability: Do the same pre-trained word embeddings result in the best performance …
ImputationWord EmbeddingsAddressing Biases in the Texts using an End-to-End Pipeline Approach
The concept of fairness is gaining popularity in academia and industry. Social media is especially vulnerable to media biases and toxic language and comments. We propose a fair ML pipeline that takes a text as input and …
FairnessWord EmbeddingsBlind signal decomposition of various word embeddings based on join and individual variance explained
In recent years, natural language processing (NLP) has become one of the most important areas with various applications in human's life. As the most fundamental task, the field of word embedding still requires more atten…
Dimensionality ReductionSentiment AnalysisWord Embeddings