paper-with-me

홈 › Papers

AdaFuse: Adaptive Multimodal Fusion for Lung Cancer Risk Prediction via Reinforcement Learning

2026-01-30 · Chongyu Qu, Zhengyi Lu, Yuxiang Lai, Thomas Z. Li, Junchao Zhu, Junlin Guo, Juming Xiong, Yanfan Zhu, Yuechen Yang, Allen J. Luna, Kim L. Sandler, Bennett A. Landman, Yuankai Huo arxiv

Multimodal fusion has emerged as a promising paradigm for disease diagnosis and prognosis, integrating complementary information from heterogeneous data sources such as medical images, clinical records, and radiology reports. However, existing fusion methods process all available modalities through the network, either treating them equally or learning to assign different contribution weights, leaving a fundamental question unaddressed: for a given patient, should certain modalities be used at all? We present AdaFuse, an adaptive multimodal fusion framework that leverages reinforcement learning (RL) to learn patient-specific modality selection and fusion strategies for lung cancer risk prediction. AdaFuse formulates multimodal fusion as a sequential decision process, where the policy network iteratively decides whether to incorporate an additional modality or proceed to prediction based on the information already acquired. This sequential formulation enables the model to condition each selection on previously observed modalities and terminate early when sufficient information is available, rather than committing to a fixed subset upfront. We evaluate AdaFuse on the National Lung Screening Trial (NLST) dataset. Experimental results demonstrate that AdaFuse achieves the highest AUC (0.762) compared to the best single-modality baseline (0.732), the best fixed fusion strategy (0.759), and adaptive baselines including DynMM (0.754) and MoE (0.742), while using fewer FLOPs than all triple-modality methods. Our work demonstrates the potential of reinforcement learning for personalized multimodal fusion in medical imaging, representing a shift from uniform fusion strategies toward adaptive diagnostic pipelines that learn when to consult additional modalities and when existing information suffices for accurate prediction.

📄 PDF Abstract BibTeX arXiv:2602.00347

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Similar Papers 제목 키워드 기반

Multimodal fusion of imaging and genomics for lung cancer recurrence prediction

2020-02-05 · Vaishnavi Subramanian, Minh N. Do, Tanveer Syeda-Mahmood

Lung cancer has a high rate of recurrence in early-stage patients. Predicting the post-surgical recurrence in lung cancer patients has traditionally been approached using single modality information of genomics or radiol…

Computed Tomography (CT)Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)

AdaFuse: Adaptive Ensemble Decoding with Test-Time Scaling for LLMs

2026-01-09 · Chengming Cui, Tianxin Wei, Ziyi Chen, Ruizhong Qiu 외 arxiv

Large language models (LLMs) exhibit complementary strengths arising from differences in pretraining data, model architectures, and decoding behaviors. Inference-time ensembling provides a practical way to combine these …

Open-Domain Question AnsweringArithmetic ReasoningMachine Translation

AdaFuse: Adaptive Multiview Fusion for Accurate Human Pose Estimation in the Wild

2020-10-26 · Zhe Zhang, Chunyu Wang, Weichao Qiu, Wenhu Qin 외

Occlusion is probably the biggest challenge for human pose estimation in the wild. Typical solutions often rely on intrusive sensors such as IMUs to detect occluded joints. To make the task truly unconstrained, we presen…

3D Human Pose EstimationPose Estimation

RadioPathomics: Multimodal Learning in Non-Small Cell Lung Cancer for Adaptive Radiotherapy

2022-04-26 · Matteo Tortora, Ermanno Cordelli, Rosa Sicilia, Lorenzo Nibid 외

The current cancer treatment practice collects multimodal data, such as radiology images, histopathology slides, genomics and clinical data. The importance of these data sources taken individually has fostered the recent…

Data Integration

AdaFuse: Adaptive Medical Image Fusion Based on Spatial-Frequential Cross Attention

2023-10-09 · Xianming Gu, Lihui Wang, Zeyu Deng, Ying Cao 외

Multi-modal medical image fusion is essential for the precise clinical diagnosis and surgical navigation since it can merge the complementary information in multi-modalities into a single image. The quality of the fused …