Enabling Training-Free Text-Based Remote Sensing Segmentation
Recent advances in Vision Language Models (VLMs) and Vision Foundation Models (VFMs) have opened new opportunities for zero-shot text-guided segmentation of remote sensing imagery. However, most existing approaches still rely on additional trainable components, limiting their generalisation and practical applicability. In this work, we investigate to what extent text-based remote sensing segmentation can be achieved without additional training, by relying solely on existing foundation models. We propose a simple yet effective approach that integrates contrastive and generative VLMs with the Segment Anything Model (SAM), enabling a fully training-free or lightweight LoRA-tuned pipeline. Our contrastive approach employs CLIP as mask selector for SAM's grid-based proposals, achieving state-of-the-art open-vocabulary semantic segmentation (OVSS) in a completely zero-shot setting. In parallel, our generative approach enables reasoning and referring segmentation by generating click prompts for SAM using GPT-5 in a zero-shot setting and a LoRA-tuned Qwen-VL model, with the latter yielding the best results. Extensive experiments across 19 remote sensing benchmarks, including open-vocabulary, referring, and reasoning-based tasks, demonstrate the strong capabilities of our approach. Code will be released at https://github.com/josesosajs/trainfree-rs-segmentation.
Code (0)
등록된 구현이 없습니다.
Tasks
Semantic SegmentationSimilar Papers 제목 키워드 기반
Beyond Open Vocabulary: Multimodal Prompting for Object Detection in Remote Sensing Images
Open-vocabulary object detection in remote sensing commonly relies on text-only prompting to specify target categories, implicitly assuming that inference-time category queries can be reliably grounded through pretrainin…
Object DetectionBeyond Pixels: A Training-Free, Text-to-Text Framework for Remote Sensing Image Retrieval
Semantic retrieval of remote sensing (RS) images is a critical task fundamentally challenged by the \textquote{semantic gap}, the discrepancy between a model's low-level visual features and high-level human concepts. Whi…
Cross-Modal RetrievalSemantic RetrievalImage RetrievalConInfer: Context-Aware Inference for Training-Free Open-Vocabulary Remote Sensing Segmentation
Training-free open-vocabulary remote sensing segmentation (OVRSS), empowered by vision-language models, has emerged as a promising paradigm for achieving category-agnostic semantic understanding in remote sensing imagery…
Semantic SegmentationSegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Images
Remote sensing image plays an irreplaceable role in fields such as agriculture, water resources, military, and disaster relief. Pixel-level interpretation is a critical aspect of remote sensing image applications; howeve…
Open Vocabulary Semantic SegmentationOpen-Vocabulary Semantic SegmentationSemantic SegmentationRSRefSeg: Referring Remote Sensing Image Segmentation with Foundation Models
Referring remote sensing image segmentation is crucial for achieving fine-grained visual understanding through free-format textual input, enabling enhanced scene and object extraction in remote sensing applications. Curr…
Image SegmentationSegmentationSemantic Segmentation