홈 › Papers

Sense Less, Infer More: Agentic Multimodal Transformers for Edge Medical Intelligence

2026-04-12 · Chengwei Zhou, Zhaoyan Jia, Haotian Yu, Xuming Chen, Brandon Lee, Christopher Pulliam, Steve Majerus, Massoud Pedram, Gourav Datta arxiv

Edge-based multimodal medical monitoring requires models that balance diagnostic accuracy with severe energy constraints. Continuous acquisition of ECG, PPG, EMG, and IMU streams rapidly drains wearable batteries, often limiting operation to under 10 hours, while existing systems overlook the high temporal redundancy present in physiological signals. We introduce Adaptive Multimodal Intelligence (AMI), an end-to-end framework that jointly learns when to sense and how to infer. AMI integrates three components: (1) a lightweight Agentic Modality Controller that uses differentiable Gumbel-Sigmoid gating to dynamically select active sensors based on model confidence and task relevance; (2) a Learned Sigma-Delta Sensing module that applies patch-wise Delta-Sigma operations with learnable thresholds to skip temporally redundant samples; and (3) a Foundation-backed Multimodal Prediction Model built on unimodal foundation encoders and a cross-modal transformer with temporal context, enabling robust fusion even under gated or missing inputs. These components are trained jointly via a multi-objective loss combining classification accuracy, sparsity regularization, cross-modal alignment, and predictive coding. AMI is hardware-aware, supporting dynamic computation graphs and masked operations, leading to real energy and latency savings. Across MHEALTH, HMC Sleep, and WESAD datasets, it reduces sensor usage by 48.8% while improving state-of-the-art accuracy by 1.9% on average.

📄 PDF Abstract BibTeX arXiv:2604.10404

Code (0)

등록된 구현이 없습니다.

SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learning

2025-12-30 · Yong Xien Chng, Tao Hu, Wenwen Tong, Xueheng Li 외 arxiv

While Vision-Language Models (VLMs) can solve complex tasks through agentic reasoning, their capabilities remain largely constrained to text-oriented chain-of-thought or isolated tool invocation. They fail to exhibit the…

Reinforcement Learning Visual Reasoning Image Cropping

Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG

2026-02-24 · Inderjeet Singh, Vikas Pahuja, Aishvariya Priya Rathina Sabapathy, Chiara Picardi 외 arxiv

Current stateless defences for multimodal agentic RAG fail to detect adversarial strategies that distribute malicious semantics across retrieval, planning, and generation components. We formulate this security challenge …

SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture

2026-05-12 · Haiwen Diao, Penghao Wu, Hanming Deng, Jiahao Wang 외 arxiv

Recent large vision-language models (VLMs) remain fundamentally constrained by a persistent dichotomy: understanding and generation are treated as distinct problems, leading to fragmented architectures, cascaded pipeline…

Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearch

2025-12-02 · Yifan Zhang, Liang Hu, Haofeng Sun, Peiyu Wang 외 arxiv

Despite recent progress in multimodal agentic systems, existing approaches often treat image manipulation and web search as disjoint capabilities, rely heavily on costly reinforcement learning, and lack planning grounded…

Reinforcement Learning Image Manipulation

When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning

2026-02-25 · Muku Akasaka, Soyeon Caren Han arxiv

Visual spatial reasoning (VSR) remains challenging for modern vision-language models (VLMs), despite advances in multimodal architectures. A common strategy is to inject additional information at inference time, such as …

Multimodal Reasoning Spatial Reasoning