Tell and Predict: Kernel Classifier Prediction for Unseen Visual Classes from Unstructured Text Descriptions
In this paper we propose a framework for predicting kernelized classifiers in the visual domain for categories with no training images where the knowledge comes from textual description about these categories. Through our optimization framework, the proposed approach is capable of embedding the class-level knowledge from the text domain as kernel classifiers in the visual domain. We also proposed a distributional semantic kernel between text descriptions which is shown to be effective in our setting. The proposed framework is not restricted to textual descriptions, and can also be applied to other forms knowledge representations. Our approach was applied for the challenging task of zero-shot learning of fine-grained categories from text descriptions of these categories.
Code (0)
등록된 구현이 없습니다.
Tasks
Zero-Shot LearningSimilar Papers 제목 키워드 기반
Open-Category Classification by Adversarial Sample Generation
In real-world classification tasks, it is difficult to collect training samples from all possible categories of the environment. Therefore, when an instance of an unseen class appears in the prediction stage, a robust cl…
ClassificationGeneral ClassificationA Multi-class Approach -- Building a Visual Classifier based on Textual Descriptions using Zero-Shot Learning
Machine Learning (ML) techniques for image classification routinely require many labelled images for training the model and while testing, we ought to use images belonging to the same domain as those used for training. I…
image-classificationImage ClassificationTransfer LearningZero-Shot LearningOut of Distribution Detection via Domain-Informed Gaussian Process State Space Models
In order for robots to safely navigate in unseen scenarios using learning-based methods, it is important to accurately detect out-of-training-distribution (OoD) situations online. Recently, Gaussian process state-space m…
NavigateOut-of-Distribution DetectionState Space ModelsWrite a Classifier: Predicting Visual Classifiers from Unstructured Text
People typically learn through exposure to visual concepts associated with linguistic descriptions. For instance, teaching visual object categories to children is often accompanied by descriptions in text or speech. In a…
regressionTransfer LearningHigher-order Pooling of CNN Features via Kernel Linearization for Action Recognition
Most successful deep learning algorithms for action recognition extend models designed for image-based tasks such as object recognition to video. Such extensions are typically trained for actions on single video frames o…
Action RecognitionFine-grained Action RecognitionObject RecognitionTemporal Action Localization