Learning Concept Taxonomies from Multi-modal Data
We study the problem of automatically building hypernym taxonomies from textual and visual data. Previous works in taxonomy induction generally ignore the increasingly prominent visual data, which encode important perceptual semantics. Instead, we propose a probabilistic model for taxonomy induction by jointly leveraging text and images. To avoid hand-crafted feature engineering, we design end-to-end features based on distributed representations of images and words. The model is discriminatively trained given a small set of existing ontologies and is capable of building full taxonomies from scratch for a collection of unseen conceptual label items with associated images. We evaluate our model and features on the WordNet hierarchies, where our system outperforms previous approaches by a large gap.
Code (0)
등록된 구현이 없습니다.
Tasks
Feature EngineeringSimilar Papers 제목 키워드 기반
Open-Linguistic Concept Unified Learning for Cross-Site Interpretable Dermatology Image Diagnosis
Human-interpretable computer-aided diagnosis is crucial for clinical decision making. Concept-based models excel by providing transparent reasoning and enabling post-hoc, clinician-in-the-loop interventions. However, the…
Decision MakingConcept Stability for Constructing Taxonomies of Web-site Users
Owners of a web-site are often interested in analysis of groups of users of their site. Information on these groups can help optimizing the structure and contents of the site. In this paper we use an approach based on fo…
Learning What You Need from What You Did: Product Taxonomy Expansion with User Behaviors Supervision
Taxonomies have been widely used in various domains to underpin numerous applications. Specially, product taxonomies serve an essential role in the e-commerce domain for the recommendation, browsing, and query understand…
Contrastive LearningGraph Neural NetworkTaxonomy ExpansionWho Should Go First? A Self-Supervised Concept Sorting Model for Improving Taxonomy Expansion
Taxonomies have been widely used in various machine learning and text mining systems to organize knowledge and facilitate downstream tasks. One critical challenge is that, as data and business scope grow in real applicat…
Taxonomy ExpansionEnhancing Taxonomy Completion with Concept Generation via Fusing Relational Representations
Automatic construction of a taxonomy supports many applications in e-commerce, web search, and question answering. Existing taxonomy expansion or completion methods assume that new concepts have been accurately extracted…
Knowledge GraphsTaxonomy Expansion