Learning Taxonomies of Concepts and not Words using Contextualized Word Representations: A Position Paper
Taxonomies are semantic hierarchies of concepts. One limitation of current taxonomy learning systems is that they define concepts as single words. This position paper argues that contextualized word representations, which recently achieved state-of-the-art results on many competitive NLP tasks, are a promising method to address this limitation. We outline a novel approach for taxonomy learning that (1) defines concepts as synsets, (2) learns density-based approximations of contextualized word representations, and (3) can measure similarity and hypernymy among them.
Code (0)
등록된 구현이 없습니다.
Tasks
PositionSimilar Papers 제목 키워드 기반
How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings
Replacing static word embeddings with contextualized word representations has yielded significant improvements on many NLP tasks. However, just how contextual are the contextualized representations produced by models suc…
Word EmbeddingsThe Impact of Word Splitting on the Semantic Content of Contextualized Word Representations
When deriving contextualized word representations from language models, a decision needs to be made on how to obtain one for out-of-vocabulary (OOV) words that are segmented into subwords. What is the best way to represe…
Semantic SimilaritySemantic Textual SimilarityEnhancing Taxonomy Completion with Concept Generation via Fusing Relational Representations
Automatic construction of a taxonomy supports many applications in e-commerce, web search, and question answering. Existing taxonomy expansion or completion methods assume that new concepts have been accurately extracted…
Knowledge GraphsTaxonomy ExpansionContextualized Weak Supervision for Text Classification
Weakly supervised text classification based on a few user-provided seed words has recently attracted much attention from researchers. Existing methods mainly generate pseudo-labels in a context-free manner (e.g., string …
ClassificationGeneral Classificationtext-classificationText ClassificationLost in Context? On the Sense-wise Variance of Contextualized Word Embeddings
Contextualized word embeddings in language models have given much advance to NLP. Intuitively, sentential information is integrated into the representation of words, which can help model polysemy. However, context sensit…
PositionSentenceWord EmbeddingsWord Sense Disambiguation