Reducing Hallucinations: Enhancing VQA for Flood Disaster Damage Assessment with Visual Contexts
The zero-shot performance of visual question answering (VQA) models relies heavily on prompts. For example, a zero-shot VQA for disaster scenarios could leverage well-designed Chain of Thought (CoT) prompts to stimulate the model's potential. However, using CoT prompts has some problems, such as causing an incorrect answer in the end due to the hallucination in the thought process. In this paper, we propose a zero-shot VQA named Flood Disaster VQA with Two-Stage Prompt (VQA-TSP). The model generates the thought process in the first stage and then uses the thought process to generate the final answer in the second stage. In particular, visual context is added in the second stage to relieve the hallucination problem that exists in the thought process. Experimental results show that our method exceeds the performance of state-of-the-art zero-shot VQA models for flood disaster scenarios in total. Our study provides a research basis for improving the performance of CoT-based zero-shot VQA.
Code (0)
등록된 구현이 없습니다.
Tasks
HallucinationQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
Transformer-based Flood Scene Segmentation for Developing Countries
Floods are large-scale natural disasters that often induce a massive number of deaths, extensive material damage, and economic turmoil. The effects are more extensive and longer-lasting in high-population and low-resourc…
Disaster ResponseFlooded Building SegmentationScene SegmentationSemantic SegmentationUnleashing the Potential of Large Language Model: Zero-shot VQA for Flood Disaster Scenario
Visual question answering (VQA) is a fundamental and essential AI task, and VQA-based disaster scenario understanding is a hot research topic. For instance, we can ask questions about a disaster image by the VQA model an…
Language ModelingLanguage ModellingLarge Language ModelMultiple-choice+3Flood-DamageSense: Multimodal Mamba with Multitask Learning for Building Flood Damage Assessment using SAR Remote Sensing Imagery
Most post-disaster damage classifiers succeed only when destructive forces leave clear spectral or structural signatures -- conditions rarely present after inundation. Consequently, existing models perform poorly at iden…
Building Damage AssessmentBuilding Flood Damage AssessmentFlooded Building SegmentationMambaFloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understanding
Visual scene understanding is the core task in making any crucial decision in any computer vision system. Although popular computer vision datasets like Cityscapes, MS-COCO, PASCAL provide good benchmarks for several tas…
image-classificationImage Classificationobject-detectionObject Detection+6FloodDamageCast: Building Flood Damage Nowcasting with Machine Learning and Data Augmentation
Near-real time estimation of damage to buildings and infrastructure, referred to as damage nowcasting in this study, is crucial for empowering emergency responders to make informed decisions regarding evacuation orders a…
Data AugmentationDisaster Response