Location-Aware Visual Question Generation with Lightweight Models
This work introduces a novel task, location-aware visual question generation (LocaVQG), which aims to generate engaging questions from data relevant to a particular geographical location. Specifically, we represent such location-aware information with surrounding images and a GPS coordinate. To tackle this task, we present a dataset generation pipeline that leverages GPT-4 to produce diverse and sophisticated questions. Then, we aim to learn a lightweight model that can address the LocaVQG task and fit on an edge device, such as a mobile phone. To this end, we propose a method which can reliably generate engaging questions from location-aware information. Our proposed method outperforms baselines regarding human evaluation (e.g., engagement, grounding, coherence) and automatic evaluation metrics (e.g., BERTScore, ROUGE-2). Moreover, we conduct extensive ablation studies to justify our proposed techniques for both generating the dataset and solving the task.
Code (1)
Tasks
Dataset GenerationQuestion GenerationQuestion-GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
InViC: Intent-aware Visual Cues for Medical Visual Question Answering
Medical visual question answering (Med-VQA) aims to answer clinically relevant questions grounded in medical images. However, existing multimodal large language models (MLLMs) often exhibit shortcut answering, producing …
Visual Question AnsweringAnswer GenerationGRACE: Grounded Reasoning via Adapter Composition and Evidence-Aware Calibration for Educational Visual Question Answering
Educational visual question answering, or VQA, requires models to solve curriculum-oriented multiple-choice questions using both language and visual evidence. Compared with conventional open-ended VQA, educational exampl…
Visual Question AnsweringLiVLR: A Lightweight Visual-Linguistic Reasoning Framework for Video Question Answering
Video Question Answering (VideoQA), aiming to correctly answer the given question based on understanding multi-modal video content, is challenging due to the rich video content. From the perspective of video understandin…
DiversityQuestion AnsweringVideo Question AnsweringVideo Understanding+1Enhancing Medical Large Vision-Language Models via Alignment Distillation
Medical Large Vision-Language Models (Med-LVLMs) have shown promising results in clinical applications, but often suffer from hallucinated outputs due to misaligned visual understanding. In this work, we identify two fun…
Visual Question AnsweringMedical Report GenerationRepresentation LearningLocation-Aware Fine-Grained Representation Learning for Medical Vision Foundation Models
Fine-grained visual representations are essential for medical image analysis, particularly when diagnostically relevant evidence is subtle and spatially localized. Modern transformer-based medical vision encoders must th…
Visual Question AnsweringSelf-Supervised LearningRepresentation LearningPhrase Grounding