SegRGB-X: General RGB-X Semantic Segmentation Model

2026-03-30 · Jiong Liu, Yingjie Xu, Xingcheng Zhou, Rui Song, Walter Zimmer, Alois Knoll, Hu Cao arxiv

Semantic segmentation across arbitrary sensor modalities faces significant challenges due to diverse sensor characteristics, and the traditional configurations for this task result in redundant development efforts. We address these challenges by introducing a universal arbitrary-modal semantic segmentation framework that unifies segmentation across multiple modalities. Our approach features three key innovations: (1) the Modality-aware CLIP (MA-CLIP), which provides modality-specific scene understanding guidance through LoRA fine-tuning; (2) Modality-aligned Embeddings for capturing fine-grained features; and (3) the Domain-specific Refinement Module (DSRM) for dynamic feature adjustment. Evaluated on five diverse datasets with different complementary modalities (event, thermal, depth, polarization, and light field), our model surpasses specialized multi-modal methods and achieves state-of-the-art performance with a mIoU of 65.03%. The codes will be released upon acceptance.

📄 PDF Abstract BibTeX arXiv:2603.28023

Code (0)

등록된 구현이 없습니다.

Tasks

Semantic Segmentation Scene Understanding

CNN-aware Binary Map for General Semantic Segmentation

2016-09-29 · Mahdyar Ravanbakhsh, Hossein Mousavi, Moin Nabi, Mohammad Rastegari 외

In this paper we introduce a novel method for general semantic segmentation that can benefit from general semantics of Convolutional Neural Network (CNN). Our segmentation proposes visually and semantically coherent imag…

Clustering Image Segmentation Segmentation Semantic Segmentation

Zero-Shot Semantic Segmentation via Spatial and Multi-Scale Aware Visual Class Embedding

2021-11-30 · Sungguk Cha, Yooseung Wang

Fully supervised semantic segmentation technologies bring a paradigm shift in scene understanding. However, the burden of expensive labeling cost remains as a challenge. To solve the cost problem, recent studies proposed…

Domain Adaptation Language Modeling Language Modelling Scene Understanding+4

Meta-Learned Feature Critics for Domain Generalized Semantic Segmentation

2021-12-27 · Zu-Yun Shiau, Wei-Wei Lin, Ci-Siang Lin, Yu-Chiang Frank Wang

How to handle domain shifts when recognizing or segmenting visual data across domains has been studied by learning and vision communities. In this paper, we address domain generalized semantic segmentation, in which the …

Disentanglement Domain Adaptation Domain Generalization Meta-Learning+2

GP-NeRF: Generalized Perception NeRF for Context-Aware 3D Scene Understanding

2023-11-20 · CVPR 2024 1 · Hao Li, Dingwen Zhang, Yalun Dai, Nian Liu 외

Applying NeRF to downstream perception tasks for scene understanding and representation is becoming increasingly popular. Most existing methods treat semantic prediction as an additional rendering task, \textit{i.e.}, th…

Instance Segmentation NeRF Scene Understanding Segmentation+1

Efficient Convolutional Neural Network with Binary Quantization Layer

2016-11-21 · Mahdyar Ravanbakhsh, Hossein Mousavi, Moin Nabi, Lucio Marcenaro 외

In this paper we introduce a novel method for segmentation that can benefit from general semantics of Convolutional Neural Network (CNN). Our segmentation proposes visually and semantically coherent image segments. We us…

Clustering Image Segmentation Quantization Segmentation+1