Language-guided Few-shot Semantic Segmentation

2023-11-23 · Jing Wang, Yuang Liu, Qiang Zhou, Fan Wang

Few-shot learning is a promising way for reducing the label cost in new categories adaptation with the guidance of a small, well labeled support set. But for few-shot semantic segmentation, the pixel-level annotations of support images are still expensive. In this paper, we propose an innovative solution to tackle the challenge of few-shot semantic segmentation using only language information, i.e.image-level text labels. Our approach involves a vision-language-driven mask distillation scheme, which contains a vision-language pretraining (VLP) model and a mask refiner, to generate high quality pseudo-semantic masks from text prompts. We additionally introduce a distributed prototype supervision method and complementary correlation matching module to guide the model in digging precise semantic relations among support and query images. The experiments on two benchmark datasets demonstrate that our method establishes a new baseline for language-guided few-shot semantic segmentation and achieves competitive results to recent vision-guided methods.

📄 PDF Abstract BibTeX arXiv:2311.13865

Code (0)

등록된 구현이 없습니다.

Tasks

Few-Shot Learning Few-Shot Semantic Segmentation Segmentation Semantic Segmentation

A Language-Guided Benchmark for Weakly Supervised Open Vocabulary Semantic Segmentation

2023-02-27 · Prashant Pandey, Mustafa Chasmai, Monish Natarajan, Brejesh lall

Increasing attention is being diverted to data-efficient problem settings like Open Vocabulary Semantic Segmentation (OVSS) which deals with segmenting an arbitrary object that may or may not be seen during training. The…

Few-Shot Semantic Segmentation Language Modelling Open Vocabulary Semantic Segmentation Open-Vocabulary Semantic Segmentation+3

Rewrite Caption Semantics: Bridging Semantic Gaps for Language-Supervised Semantic Segmentation

2023-09-24 · NeurIPS 2023 11 · Yun Xing, Jian Kang, Aoran Xiao, Jiahao Nie 외

Vision-Language Pre-training has demonstrated its remarkable zero-shot recognition ability and potential to learn generalizable visual representations from language supervision. Taking a step ahead, language-supervised s…

Segmentation Semantic Segmentation Zero-Shot Learning

Self-guided Few-shot Semantic Segmentation for Remote Sensing Imagery Based on Large Vision Models

2023-11-22 · Xiyu Qi, Yifan Wu, Yongqiang Mao, Wenhui Zhang 외

The Segment Anything Model (SAM) exhibits remarkable versatility and zero-shot learning abilities, owing largely to its extensive training data (SA-1B). Recognizing SAM's dependency on manual guidance given its category-…

Few-Shot Semantic Segmentation Prompt Learning Segmentation Semantic Segmentation+1

AG-VAS: Anchor-Guided Zero-Shot Visual Anomaly Segmentation with Large Multimodal Models

2026-03-01 · Zhen Qu, Xian Tao, Xiaoyi Bao, Dingrong Wang 외 arxiv

Large multimodal models (LMMs) exhibit strong task generalization capabilities, offering new opportunities for zero-shot visual anomaly segmentation (ZSAS). However, existing LMM-based segmentation approaches still face …

Show or Tell? Effectively prompting Vision-Language Models for semantic segmentation

2025-03-25 · Niccolo Avogaro, Thomas Frick, Mattia Rigotti, Andrea Bartezzaghi 외

Large Vision-Language Models (VLMs) are increasingly being regarded as foundation models that can be instructed to solve diverse tasks by prompting, without task-specific training. We examine the seemingly obvious questi…

Few-Shot Learning Segmentation Semantic Segmentation