Evaluating and Enhancing Negation Comprehension in Remote Sensing MLLMs
Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in various Remote Sensing (RS) tasks. However, their ability to comprehend negation remains underexplored, limiting deployment in real-world applications where models must explicitly identify what is false or absent, e.g., emergency responders need to locate non-flooded routes for evacuation. To comprehensively study this limitation, we introduce RS-Neg, the first benchmark to evaluate negation understanding across region-level to scene-level tasks. Specifically, we design an automated data generation pipeline for RS imagery, using LLMs to synthesize diverse negation queries, and introduce a dynamic visual focus module for verification. Our evaluation reveals that advanced RS MLLMs struggle with negation, exhibiting hallucinations and substantial performance degradation. To close this gap, we propose NeFo, a novel test-time learning method that explicitly incorporates the logical role of negation into the model optimization. Remarkably, using about 5\% unlabeled test samples, NeFo significantly improves the negation understanding of models and shows strong generalization to unseen tasks.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Thunder-KoNUBench: A Corpus-Aligned Benchmark for Korean Negation Understanding
Although negation is known to challenge large language models (LLMs), benchmarks for evaluating negation understanding-especially in Korean-are scarce. We conduct a corpus-based analysis of Korean negation and show that …
SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding
Remote Sensing Large Multi-Modal Models (RSLMMs) are developing rapidly and showcase significant capabilities in remote sensing imagery (RSI) comprehension. However, due to the limitations of existing datasets, RSLMMs ha…
Graph GenerationRelationScene Graph GenerationU-Net Ensemble for Enhanced Semantic Segmentation in Remote Sensing Imagery
Semantic segmentation of remote sensing imagery stands as a fundamental task within the domains of both remote sensing and computer vision. Its objective is to generate a comprehensive pixel-wise segmentation map of an i…
SegmentationSegmentation Of Remote Sensing ImagerySemantic SegmentationRSRefSeg: Referring Remote Sensing Image Segmentation with Foundation Models
Referring remote sensing image segmentation is crucial for achieving fine-grained visual understanding through free-format textual input, enabling enhanced scene and object extraction in remote sensing applications. Curr…
Image SegmentationSegmentationSemantic SegmentationAquila: A Hierarchically Aligned Visual-Language Model for Enhanced Remote Sensing Image Comprehension
Recently, large vision language models (VLMs) have made significant strides in visual language capabilities through visual instruction tuning, showing great promise in the field of remote sensing image interpretation. Ho…
Image ComprehensionLanguage ModelingLanguage ModellingLarge Language Model