Probing Taxonomic and Thematic Embeddings for Taxonomic Information
Modelling taxonomic and thematic relatedness is important for building AI with comprehensive natural language understanding. The goal of this paper is to learn more about how taxonomic information is structurally encoded in embeddings. To do this, we design a new hypernym-hyponym probing task and perform a comparative probing study of taxonomic and thematic SGNS and GloVe embeddings. Our experiments indicate that both types of embeddings encode some taxonomic information, but the amount, as well as the geometric properties of the encodings, are independently related to both the encoder architecture, as well as the embedding training data. Specifically, we find that only taxonomic embeddings carry taxonomic information in their norm, which is determined by the underlying distribution in the data.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language UnderstandingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Semantic Relatedness and Taxonomic Word Embeddings
This paper connects a series of papers dealing with taxonomic word embeddings. It begins by noting that there are different types of semantic relatedness and that different lexical representations encode different forms …
Word EmbeddingsImproved Knowledge Graph Embedding using Background Taxonomic Information
Knowledge graphs are used to represent relational information in terms of triples. To enable learning about domains, embedding models, such as tensor factorization models, can be used to make predictions of new triples. …
Graph EmbeddingKnowledge Graph CompletionKnowledge Graph EmbeddingKnowledge GraphsModeling Label Space Interactions in Multi-label Classification using Box Embeddings
Multi-label classification is a challenging structured prediction task in which a set of output class labels are predicted for each input. Real-world datasets often have natural or latent taxonomic relationships between …
ClassificationHierarchical Multi-label ClassificationInductive BiasMulti-Label Classification+2English WordNet Random Walk Pseudo-Corpora
This is a resource description paper that describes the creation and properties of a set of pseudo-corpora generated artificially from a random walk over the English WordNet taxonomy. Our WordNet taxonomic random walk im…
Word Embeddings