Taming SAM3 in the Wild: A Concept Bank for Open-Vocabulary Segmentation

2026-02-06 · Gensheng Pei, Xiruo Jiang, Yazhou Yao, Xiangbo Shu, Fumin Shen, Byeungwoo Jeon arxiv

The recent introduction of \texttt{SAM3} has revolutionized Open-Vocabulary Segmentation (OVS) through \textit{promptable concept segmentation}, which grounds pixel predictions in flexible concept prompts. However, this reliance on pre-defined concepts makes the model vulnerable: when visual distributions shift (\textit{data drift}) or conditional label distributions evolve (\textit{concept drift}) in the target domain, the alignment between visual evidence and prompts breaks down. In this work, we present \textsc{ConceptBank}, a parameter-free calibration framework to restore this alignment on the fly. Instead of adhering to static prompts, we construct a dataset-specific concept bank from the target statistics. Our approach (\textit{i}) anchors target-domain evidence via class-wise visual prototypes, (\textit{ii}) mines representative supports to suppress outliers under data drift, and (\textit{iii}) fuses candidate concepts to rectify concept drift. We demonstrate that \textsc{ConceptBank} effectively adapts \texttt{SAM3} to distribution drifts, including challenging natural-scene and remote-sensing scenarios, establishing a new baseline for robustness and efficiency in OVS. Code and model are available at https://github.com/pgsmall/ConceptBank.

📄 PDF Abstract BibTeX arXiv:2602.06333

Code (0)

등록된 구현이 없습니다.

Open Vocabulary Scene Parsing

2017-03-26 · ICCV 2017 10 · Hang Zhao, Xavier Puig, Bolei Zhou, Sanja Fidler 외

Recognizing arbitrary objects in the wild has been a challenging problem due to the limitations of existing classification models and datasets. In this paper, we propose a new task that aims at parsing scenes with a larg…

General Classification Scene Parsing

Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models

2023-03-08 · CVPR 2023 1 · Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon 외

We present ODISE: Open-vocabulary DIffusion-based panoptic SEgmentation, which unifies pre-trained text-image diffusion and discriminative models to perform open-vocabulary panoptic segmentation. Text-to-image diffusion …

Open Vocabulary Panoptic Segmentation Open Vocabulary Semantic Segmentation Open-World Instance Segmentation Panoptic Segmentation+3

Improving Interpretable Embeddings for Ad-hoc Video Search with Generative Captions and Multi-word Concept Bank

2024-04-09 · Jiaxin Wu, Chong-Wah Ngo, Wing-Kwong Chan

Aligning a user query and video clips in cross-modal latent space and that with semantic concepts are two mainstream approaches for ad-hoc video search (AVS). However, the effectiveness of existing approaches is bottlene…

Ad-hoc video search

Contextual Earnings-22: A Speech Recognition Benchmark with Custom Vocabulary in the Wild

2026-03-28 · Berkin Durmus, Chen Cen, Eduardo Pacheco, Arda Okan 외 arxiv

The accuracy frontier of speech-to-text systems has plateaued on academic benchmarks.1 In contrast, industrial benchmarks and adoption in high-stakes domains suggest otherwise. We hypothesize that the primary difference …

Speech Recognition

WildOS: Open-Vocabulary Object Search in the Wild

2026-02-22 · Hardik Shah, Erica Tevere, Deegan Atha, Marcel Kaufmann 외 arxiv

Autonomous navigation in complex, unstructured outdoor environments requires robots to operate over long ranges without prior maps and limited depth sensing. In such settings, relying solely on geometric frontiers for ex…

Visual Reasoning