SURE: Synergistic Uncertainty-aware Reasoning for Multimodal Emotion Recognition in Conversations

2026-04-02 · Yiqiang Cai, Chengyan Wu, Bolei Ma, Bo Chen, Yun Xue, Julia Hirschberg, Ziwei Gong arxiv

Multimodal emotion recognition in conversations (MERC) requires integrating multimodal signals while being robust to noise and modeling contextual reasoning. Existing approaches often emphasize fusion but overlook uncertainty in noisy features and fine-grained reasoning. We propose SURE (Synergistic Uncertainty-aware REasoning) for MERC, a framework that improves robustness and contextual modeling. SURE consists of three components: an Uncertainty-Aware Mixture-of-Experts module to handle modality-specific noise, an Iterative Reasoning module for multi-turn reasoning over context, and a Transformer Gate module to capture intra- and inter-modal interactions. Experiments on benchmark MERC datasets show that SURE consistently outperforms state-of-the-art methods, demonstrating its effectiveness in robust multimodal reasoning. These results highlight the importance of uncertainty modeling and iterative reasoning in advancing emotion recognition in conversational settings.

📄 PDF Abstract BibTeX arXiv:2604.01916

Code (0)

등록된 구현이 없습니다.

Tasks

Multimodal Emotion Recognition Multimodal Reasoning

MedAlign: A Synergistic Framework of Multimodal Preference Optimization and Federated Meta-Cognitive Reasoning

2025-10-24 · Siyong Chen, Jinbo Wen, Jiawen Kang, Tenghui Huang 외 arxiv

Recently, large models have shown significant potential for smart healthcare. However, the deployment of Large Vision-Language Models (LVLMs) for clinical services is currently hindered by three critical challenges: a te…

Visual Question Answering

When Safe Unimodal Inputs Collide: Optimizing Reasoning Chains for Cross-Modal Safety in Multimodal Large Language Models

2025-09-15 · Wei Cai, Shujuan Liu, Jian Zhao, Ziyan Shi 외 arxiv

Multimodal Large Language Models (MLLMs) are susceptible to the implicit reasoning risk, wherein innocuous unimodal inputs synergistically assemble into risky multimodal data that produce harmful outputs. We attribute th…

Enhancing Multi-Modal LLMs Reasoning via Difficulty-Aware Group Normalization

2026-02-25 · Jinghan Li, Junfeng Fang, Jinda Lu, Yuan Wang 외 arxiv

Reinforcement Learning with Verifiable Rewards (RLVR) and Group Relative Policy Optimization (GRPO) have significantly advanced the reasoning capabilities of large language models. Extending these methods to multimodal s…

Reinforcement Learning Multimodal Reasoning

URMF: Uncertainty-aware Robust Multimodal Fusion for Multimodal Sarcasm Detection

2026-04-08 · Zhenyu Wang, Weichen Cheng, Weijia Li, Junjie Mou 외 arxiv

Multimodal sarcasm detection (MSD) aims to identify sarcastic intent from semantic incongruity between text and image. Although recent methods have improved MSD through cross-modal interaction and incongruity reasoning, …

Contrastive Learning Sarcasm Detection

See the Forest and the Trees: A Synergistic Reasoning Framework for Knowledge-Based Visual Question Answering

2025-07-23 · Junjie Wang, Yunhan Tang, Yijie Wang, Zhihao Yuan 외 arxiv

Multimodal Large Language Models (MLLMs) have pushed the frontiers of Knowledge-Based Visual Question Answering (KBVQA), yet their reasoning is fundamentally bottlenecked by a reliance on uni-dimensional evidence. This "…