paper-with-me

홈 › Papers

Synergy vs. Noise: Performance-Guided Multimodal Fusion For Biochemical Recurrence-Free Survival in Prostate Cancer

2025-11-14 · Seth Alain Chang, Muhammad Mueez Amjad, Noorul Wahab, Ethar Alzaid, Nasir Rajpoot, Adam Shephard arxiv

Multimodal deep learning (MDL) has emerged as a transformative approach in computational pathology. By integrating complementary information from multiple data sources, MDL models have demonstrated superior predictive performance across diverse clinical tasks compared to unimodal models. However, the assumption that combining modalities inherently improves performance remains largely unexamined. We hypothesise that multimodal gains depend critically on the predictive quality of individual modalities, and that integrating weak modalities may introduce noise rather than complementary information. We test this hypothesis on a prostate cancer dataset with histopathology, radiology, and clinical data to predict time-to-biochemical recurrence. Our results confirm that combining high-performing modalities yield superior performance compared to unimodal approaches. However, integrating a poor-performing modality with other higher-performing modalities degrades predictive accuracy. These findings demonstrate that multimodal benefit requires selective, performance-guided integration rather than indiscriminate modality combination, with implications for MDL design across computational pathology and medical imaging.

📄 PDF Abstract BibTeX arXiv:2511.11452

Code (0)

등록된 구현이 없습니다.

Tasks

Multimodal Deep Learning

Similar Papers 제목 키워드 기반

Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information Decomposition

2026-05-31 · Wanlong Fang, Tianle Zhang, Wen Tao, Alvin Chan arxiv

Understanding modality interaction in multimodal large language models (MLLMs) is central to reliable deployment. We introduce Partial Information Decomposition (PID) as a decision-level framework that separates unique, …

Multimodal Reasoning

Adversarial-Guided Diffusion for Multimodal LLM Attacks

2025-07-31 · Chengwei Xia, Fan Ma, Ruijie Quan, Kun Zhan 외 arxiv

This paper addresses the challenge of generating adversarial image using a diffusion model to deceive multimodal large language models (MLLMs) into generating the targeted responses, while avoiding significant distortion…

Adversarial Attack

Similarity Guided Multimodal Fusion Transformer for Semantic Location Prediction in Social Media

2024-05-09 · Zhizhen Zhang, Ning Wang, Haojie Li, Zhihui Wang

Semantic location prediction aims to derive meaningful location insights from multimodal social media posts, offering a more contextual understanding of daily activities than using GPS coordinates. This task faces signif…

Language Modelling

MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts for Multimodal Knowledge Graph Completion

2026-07-17 · Xu Hou, Meiyu Liang, Wei Huang, Yawen Li 외 arxiv

Multimodal Knowledge Graph Completion (MKGC) requires inferring missing entities from structural, textual, and visual cues. Existing diffusion-based MKGC methods usually denoise directly on raw multimodal features. Such …

Knowledge Graph Completion

Fusion-E2Pulse: A Multimodal Event-RGB Fusion Network for Non-contact Pulse Wave Reconstruction

2026-06-14 · Qian Feng, Hao Guo, Yan Niu, Zhenhuan Xu 외 arxiv

Non-contact pulse wave reconstruction hinges on the precise recovery of waveform morphology, including the dicrotic notch. Conventional Red-Green-Blue (RGB)-based methods, which extract physiological signals from recorde…

Heart rate estimation