Overcoming Poor Word Embeddings with Word Definitions
Modern natural language understanding models depend on pretrained subword embeddings, but applications may need to reason about words that were never or rarely seen during pretraining. We show that examples that depend critically on a rarer word are more challenging for natural language inference models. Then we explore how a model could learn to use definitions, provided in natural text, to overcome this handicap. Our model's understanding of a definition is usually weaker than a well-modeled word embedding, but it recovers most of the performance gap from using a completely untrained word.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language InferenceNatural Language UnderstandingWord EmbeddingsSimilar Papers 제목 키워드 기반
Learning Word Sense Embeddings from Word Sense Definitions
Word embeddings play a significant role in many modern NLP systems. Since learning one representation per word is problematic for polysemous words and homonymous words, researchers propose to use one embedding per word s…
Word EmbeddingsWord Sense DisambiguationWord SimilarityZero-shot Word Sense Disambiguation using Sense Definition Embeddings
Word Sense Disambiguation (WSD) is a long-standing but open problem in Natural Language Processing (NLP). WSD corpora are typically small in size, owing to an expensive annotation process. Current supervised WSD methods …
Generalized Zero-Shot LearningGraph EmbeddingKnowledge Graph EmbeddingSentence+2Bridging the Defined and the Defining: Exploiting Implicit Lexical Semantic Relations in Definition Modeling
Definition modeling includes acquiring word embeddings from dictionary definitions and generating definitions of words. While the meanings of defining words are important in dictionary definitions, it is crucial to captu…
Word EmbeddingsWord Definitions from Large Language Models
Dictionary definitions are historically the arbitrator of what words mean, but this primacy has come under threat by recent progress in NLP, including word embeddings and generative models like ChatGPT. We present an exp…
Word EmbeddingsSemeval-2022 Task 1: CODWOE – Comparing Dictionaries and Word Embeddings
Word embeddings have advanced the state of the art in NLP across numerous tasks. Understanding the contents of dense neural representations is of utmost interest to the computational semantics community. We propose to fo…
Word Embeddings