Learning by Asking Questions for Knowledge-based Novel Object Recognition
In real-world object recognition, there are numerous object classes to be recognized. Conventional image recognition based on supervised learning can only recognize object classes that exist in the training data, and thus has limited applicability in the real world. On the other hand, humans can recognize novel objects by asking questions and acquiring knowledge about them. Inspired by this, we study a framework for acquiring external knowledge through question generation that would help the model instantly recognize novel objects. Our pipeline consists of two components: the Object Classifier, which performs knowledge-based object recognition, and the Question Generator, which generates knowledge-aware questions to acquire novel knowledge. We also propose a question generation strategy based on the confidence of the knowledge-aware prediction of the Object Classifier. To train the Question Generator, we construct a dataset that contains knowledge-aware questions about objects in the images. Our experiments show that the proposed pipeline effectively acquires knowledge about novel objects compared to several baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
ObjectObject RecognitionQuestion GenerationQuestion-GenerationSimilar Papers 제목 키워드 기반
Visual Question Generation for Class Acquisition of Unknown Objects
Traditional image recognition methods only consider objects belonging to already learned classes. However, since training a recognition model with every object class in the world is unfeasible, a way of getting informati…
Question GenerationQuestion-GenerationHow Well Do Large Language Models Understand Syntax? An Evaluation by Asking Natural Language Questions
While recent advancements in large language models (LLMs) bring us closer to achieving artificial general intelligence, the question persists: Do LLMs truly understand language, or do they merely mimic comprehension thro…
Prepositional Phrase AttachmentQuestion AnsweringSentenceData-free Multi-label Image Recognition via LLM-powered Prompt Tuning
This paper proposes a novel framework for multi-label image recognition without any training data, called data-free framework, which uses knowledge of pre-trained Large Language Model (LLM) to learn prompts to adapt pret…
Language ModelingLanguage ModellingLarge Language ModelMulti-Label Image Recognition+1A New Dialogue Response Generation Agent for Large Language Models by Asking Questions to Detect User's Intentions
Large Language Models (LLMs), such as ChatGPT, have recently been applied to various NLP tasks due to its open-domain generation capabilities. However, there are two issues with applying LLMs to dialogue tasks. 1. During…
Question GenerationQuestion-GenerationResponse GenerationVisual Dialogue State Tracking for Question Generation
GuessWhat?! is a visual dialogue task between a guesser and an oracle. The guesser aims to locate an object supposed by the oracle oneself in an image by asking a sequence of Yes/No questions. Asking proper questions wit…
Dialogue State TrackingQuestion GenerationQuestion-GenerationVisual Dialog