Insights into Analogy Completion from the Biomedical Domain
Analogy completion has been a popular task in recent years for evaluating the semantic properties of word embeddings, but the standard methodology makes a number of assumptions about analogies that do not always hold, either in recent benchmark datasets or when expanding into other domains. Through an analysis of analogies in the biomedical domain, we identify three assumptions: that of a Single Answer for any given analogy, that the pairs involved describe the Same Relationship, and that each pair is Informative with respect to the other. We propose modifying the standard methodology to relax these assumptions by allowing for multiple correct answers, reporting MAP and MRR in addition to accuracy, and using multiple example pairs. We further present BMASS, a novel dataset for evaluating linguistic regularities in biomedical embeddings, and demonstrate that the relationships described in the dataset pose significant semantic challenges to current word embedding methods.
Code (1)
Tasks
Word EmbeddingsSimilar Papers 제목 키워드 기반
Jointly Embedding Entities and Text with Distant Supervision
Learning representations for knowledge base entities and concepts is becoming increasingly important for NLP applications. However, recent entity embedding methods have relied on structured resources that are expensive t…
Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study
Biomedical knowledge graphs (KGs) hold rich information on entities such as diseases, drugs, and genes. Predicting missing links in these graphs can boost many important applications, such as drug design and repurposing.…
Drug DesignKnowledge Base CompletionKnowledge GraphsLink PredictionWhat Analogies Reveal about Word Vectors and their Compositionality
Analogy completion via vector arithmetic has become a common means of demonstrating the compositionality of word embeddings. Previous work have shown that this strategy works more reliably for certain types of analogical…
Semantic Textual SimilarityWord EmbeddingsImproving Biomedical Analogical Retrieval with Embedding of Structural Dependencies
Inferring the nature of the relationships between biomedical entities from text is an important problem due to the difficulty of maintaining human-curated knowledge bases in rapidly evolving fields. Neural word embedding…
RetrievalWord EmbeddingsUnderstanding Narratives through Dimensions of Analogy
Analogical reasoning is a powerful qualitative reasoning tool that enables humans to connect two situations, and to generalize their knowledge from familiar to novel situations. Cognitive Science research provides valuab…