Visual Question Generation for Class Acquisition of Unknown Objects
Traditional image recognition methods only consider objects belonging to already learned classes. However, since training a recognition model with every object class in the world is unfeasible, a way of getting information on unknown objects (i.e., objects whose class has not been learned) is necessary. A way for an image recognition system to learn new classes could be asking a human about objects that are unknown. In this paper, we propose a method for generating questions about unknown objects in an image, as means to get information about classes that have not been learned. Our method consists of a module for proposing objects, a module for identifying unknown objects, and a module for generating questions about unknown objects. The experimental results via human evaluation show that our method can successfully get information about unknown objects in an image dataset. Our code and dataset are available at https://github.com/mil-tokyo/vqg-unknown.
Code (1)
Tasks
Question GenerationQuestion-GenerationSimilar Papers 제목 키워드 기반
K-VQG: Knowledge-aware Visual Question Generation for Common-sense Acquisition
Visual Question Generation (VQG) is a task to generate questions from images. When humans ask questions about an image, their goal is often to acquire some new knowledge. However, existing studies on VQG have mainly addr…
Common Sense ReasoningQuestion GenerationQuestion-GenerationBringing personalized learning into computer-aided question generation
This paper proposes a novel and statistical method of ability estimation based on acquisition distribution for a personalized computer aided question generation. This method captures the learning outcomes over time and p…
Question GenerationQuestion-GenerationNon-monotonic Logical Reasoning Guiding Deep Learning for Explainable Visual Question Answering
State of the art algorithms for many pattern recognition problems rely on deep network models. Training these models requires a large labeled dataset and considerable computational resources. Also, it is difficult to und…
Inductive LearningLogical ReasoningQuestion AnsweringVisual Question Answering+1From Known to the Unknown: Transferring Knowledge to Answer Questions about Novel Visual and Semantic Concepts
Current Visual Question Answering (VQA) systems can answer intelligent questions about `Known' visual content. However, their performance drops significantly when questions about visually and linguistically `Unknown' con…
Novel ConceptsQuestion AnsweringRetrievalVisual Question Answering+1Solve Optimization Problems with Unknown Constraint Networks
In most optimization problems, users have a clear understanding of the function to optimize (e.g., minimize the makespan for scheduling problems). However, the constraints may be difficult to state and their modelling of…
Scheduling