SiMO: Single-Modality-Operable Multimodal Collaborative Perception

2026-03-09 · Jiageng Wen, Shengjie Zhao, Bing Li, Jiafeng Huang, Kenan Ye, Hao Deng arxiv

Collaborative perception integrates multi-agent perspectives to enhance the sensing range and overcome occlusion issues. While existing multimodal approaches leverage complementary sensors to improve performance, they are highly prone to failure--especially when a key sensor like LiDAR is unavailable. The root cause is that feature fusion leads to semantic mismatches between single-modality features and the downstream modules. This paper addresses this challenge for the first time in the field of collaborative perception, introducing Single-Modality-Operable Multimodal Collaborative Perception (SiMO). By adopting the proposed Length-Adaptive Multi-Modal Fusion (LAMMA), SiMO can adaptively handle remaining modal features during modal failures while maintaining consistency of the semantic space. Additionally, leveraging the innovative "Pretrain-Align-Fuse-RD" training strategy, SiMO addresses the issue of modality competition--generally overlooked by existing methods--ensuring the independence of each individual modality branch. Experiments demonstrate that SiMO effectively aligns multimodal features while simultaneously preserving modality-specific features, enabling it to maintain optimal performance across all individual modalities. The implementation details can be found in https://github.com/dempsey-wen/SiMO.

📄 PDF Abstract BibTeX arXiv:2603.08240

Code (0)

등록된 구현이 없습니다.

From Complexity to Parsimony: Integrating Latent Class Analysis to Uncover Multimodal Learning Patterns in Collaborative Learning

2024-11-23 · Lixiang Yan, Dragan Gašević, Linxuan Zhao, Vanessa Echeverria 외

Multimodal Learning Analytics (MMLA) leverages advanced sensing technologies and artificial intelligence to capture complex learning processes, but integrating diverse data sources into cohesive insights remains challeng…

Modeling Collaborative Multimodal Behavior in Group Dialogues: The MULTISIMO Corpus

2018-05-01 · LREC 2018 5 · Maria Koutsombogera, Carl Vogel

A Multimodal Single-Branch Embedding Network for Recommendation in Cold-Start and Missing Modality Scenarios

2024-09-26 · Christian Ganhör, Marta Moscati, Anna Hausberger, Shah Nawaz 외

Most recommender systems adopt collaborative filtering (CF) and provide recommendations based on past collective interactions. Therefore, the performance of CF algorithms degrades when few or no interactions are availabl…

Collaborative Filtering Multimodal Recommendation Recommendation Systems

Exploring Interactions and Regulations in Collaborative Learning: An Interdisciplinary Multimodal Dataset

2022-10-11 · Yante Li, Yang Liu, KhÁnh Nguyen, Henglin Shi 외

Collaborative learning is an educational approach that enhances learning through shared goals and working together. Interaction and regulation are two essential factors related to the success of collaborative learning. S…

MCMoE: Completing Missing Modalities with Mixture of Experts for Incomplete Multimodal Action Quality Assessment

2025-11-21 · Huangbiao Xu, Huanqi Wu, Xiao Ke, Junyi Wu 외 arxiv

Multimodal Action Quality Assessment (AQA) has recently emerged as a promising paradigm. By leveraging complementary information across shared contextual cues, it enhances the discriminative evaluation of subtle intra-cl…

Action Quality Assessment Representation Learning