Learning Bias-reduced Word Embeddings Using Dictionary Definitions
Pre-trained word embeddings, such as GloVe, have shown undesirable gender, racial, and religious biases. To address this problem, we propose DD-GloVe, a train-time debiasing algorithm to learn word embeddings by leveraging \underline{d}ictionary \underline{d}efinitions. We introduce dictionary-guided loss functions that encourage word embeddings to be similar to their relatively neutral dictionary definition representations. Existing debiasing algorithms typically need a pre-compiled list of seed words to represent the bias direction, along which biased information gets removed. Producing this list involves subjective decisions and it might be difficult to obtain for some types of biases. We automate the process of finding seed words: our algorithm starts from a single pair of initial seed words and automatically finds more words whose definitions display similar attributes traits. We demonstrate the effectiveness of our approach with benchmark evaluations and empirical analyses. Our code is available at https://github.com/haozhe-an/DD-GloVe.
Code (1)
Tasks
Word EmbeddingsSimilar Papers 제목 키워드 기반
Def2Vec: Extensible Word Embeddings from Dictionary Definitions
Def2Vec introduces a novel paradigm for word embeddings, leveraging dictionary definitions to learn semantic representations. By constructing term-document matrices from definitions and applying Latent Semantic Analysis …
Chunkingnamed-entity-recognitionNamed Entity RecognitionNER+5Dictionary-based Debiasing of Pre-trained Word Embeddings
Word embeddings trained on large corpora have shown to encode high levels of unfair discriminatory gender, racial, religious and ethnic biases. In contrast, human-written dictionaries describe the meanings of words in a …
Word EmbeddingsBridging the Defined and the Defining: Exploiting Implicit Lexical Semantic Relations in Definition Modeling
Definition modeling includes acquiring word embeddings from dictionary definitions and generating definitions of words. While the meanings of defining words are important in dictionary definitions, it is crucial to captu…
Word EmbeddingsAuto-Encoding Dictionary Definitions into Consistent Word Embeddings
Monolingual dictionaries are widespread and semantically rich resources. This paper presents a simple model that learns to compute word embeddings by processing dictionary definitions and trying to reconstruct them. It e…
Document ClassificationMachine TranslationSemantic SimilaritySemantic Textual Similarity+1Word Definitions from Large Language Models
Dictionary definitions are historically the arbitrator of what words mean, but this primacy has come under threat by recent progress in NLP, including word embeddings and generative models like ChatGPT. We present an exp…
Word Embeddings