홈 › Papers

Less Is More? Selective Visual Attention to High-Importance Regions for Multimodal Radiology Summarization

2026-03-31 · Mst. Fahmida Sultana Naznin, Adnan Ibney Faruq, Mushfiqur Rahman, Niloy Kumar Mondal, Md. Mehedi Hasan Shawon, Md Rakibul Hasan arxiv

Automated radiology report summarization aims to distill verbose findings into concise clinical impressions, but existing multimodal models often struggle with visual noise and fail to meaningfully improve over strong text-only baselines in the FINDINGS $\to$ IMPRESSION transformation. We challenge two prevailing assumptions: (1) that more visual input is always better, and (2) that multimodal models add limited value when findings already contain rich image-derived detail. Through controlled ablations on MIMIC-CXR benchmark, we show that selectively focusing on pathology-relevant visual patches rather than full images yields substantially better performance. We introduce ViTAS, Visual-Text Attention Summarizer, a multi-stage pipeline that combines ensemble-guided MedSAM2 lung segmentation, bidirectional cross-attention for multi-view fusion, Shapley-guided adaptive patch clustering, and hierarchical visual tokenization feeding a ViT. ViTAS achieves SOTA results with 29.25% BLEU-4 and 69.83% ROUGE-L, improved factual alignment in qualitative analysis, and the highest expert-rated human evaluation scores. Our findings demonstrate that less but more relevant visual input is not only sufficient but superior for multimodal radiology summarization.

📄 PDF Abstract BibTeX arXiv:2603.29901

Code (0)

등록된 구현이 없습니다.

Selective Particle Attention: Visual Feature-Based Attention in Deep Reinforcement Learning

2020-08-26 · Sam Blakeman, Denis Mareschal

The human brain uses selective attention to filter perceptual input so that only the components that are useful for behaviour are processed using its limited computational resources. We focus on one particular form of vi…

Deep Reinforcement Learning Multiple-choice Open-Ended Question Answering reinforcement-learning+2

An initial attempt of combining visual selective attention with deep reinforcement learning

2018-11-11 · Liu Yuezhang, Ruohan Zhang, Dana H. Ballard

Visual attention serves as a means of feature selection mechanism in the perceptual system. Motivated by Broadbent's leaky filter model of selective attention, we evaluate how such mechanism could be implemented and affe…

Atari Games Deep Reinforcement Learning feature selection Optical Flow Estimation+3

Selective Aggregation of Attention Maps Improves Diffusion-Based Visual Interpretation

2026-04-07 · Jungwon Park, Jungmin Ko, Dongnam Byun, Wonjong Rhee arxiv

Numerous studies on text-to-image (T2I) generative models have utilized cross-attention maps to boost application performance and interpret model behavior. However, the distinct characteristics of attention maps from dif…

EEG-based Decoding of Selective Visual Attention in Superimposed Videos

2024-09-19 · Yuanyuan Yao, Wout De Swaef, Simon Geirnaert, Alexander Bertrand

Selective attention enables humans to efficiently process visual stimuli by enhancing important elements and filtering out irrelevant information. Locating visual attention is fundamental in neuroscience with potential a…

Decoder EEG

Look Less, Reason More: Block-wise Attention Skipping for Efficient Multimodal LLMs

2026-06-07 · Jie Ma, Zhike Qiu, Jiayi Ji, Xiaoshuai Sun 외 arxiv

Multimodal Large Language Models (MLLMs) face a significant inference bottleneck due to the quadratic computational cost of self-attention over long visual token sequences. However, we identify a critical inefficiency in…