paper-with-me

홈 › Papers

Benchmarking Egocentric Clinical Intent Understanding Capability for Medical Multimodal Large Language Models

2026-01-11 · Shaonan Liu, Guo Yu, Xiaoling Luo, Shiyi Zheng, Wenting Chen, Jie Liu, Linlin Shen arxiv

Medical Multimodal Large Language Models (Med-MLLMs) require egocentric clinical intent understanding for real-world deployment, yet existing benchmarks fail to evaluate this critical capability. To address these challenges, we introduce MedGaze-Bench, the first benchmark leveraging clinician gaze as a Cognitive Cursor to assess intent understanding across surgery, emergency simulation, and diagnostic interpretation. Our benchmark addresses three fundamental challenges: visual homogeneity of anatomical structures, strict temporal-causal dependencies in clinical workflows, and implicit adherence to safety protocols. We propose a Three-Dimensional Clinical Intent Framework evaluating: (1) Spatial Intent: discriminating precise targets amid visual noise, (2) Temporal Intent: inferring causal rationale through retrospective and prospective reasoning, and (3) Standard Intent: verifying protocol compliance through safety checks. Beyond accuracy metrics, we introduce Trap QA mechanisms to stress-test clinical reliability by penalizing hallucinations and cognitive sycophancy. Experiments reveal current MLLMs struggle with egocentric intent due to over-reliance on global features, leading to fabricated observations and uncritical acceptance of invalid instructions.

📄 PDF Abstract BibTeX arXiv:2601.06750

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

In the Eye of MLLM: Benchmarking Egocentric Video Intent Understanding with Gaze-Guided Prompting

2025-09-09 · Taiying Peng, Jiacheng Hua, Miao Liu, Feng Lu arxiv

The emergence of advanced multimodal large language models (MLLMs) has significantly enhanced AI assistants' ability to process complex information across modalities. Recently, egocentric videos, by directly capturing us…

Video Question AnsweringGaze Estimation

EgoIntent: An Egocentric Step-level Benchmark for Understanding What, Why, and Next

2026-03-12 · Ye Pan, Chi Kit Wong, Yuanhuiyi Lyu, Hanqian Li 외 arxiv

Multimodal Large Language Models (MLLMs) have demonstrated remarkable video reasoning capabilities across diverse tasks. However, their ability to understand human intent at a fine-grained level in egocentric videos rema…

Towards Context-Aware Clinical Motion Understanding in Daily Living at Home: Freezing of Gait Detection with Egocentric Vision

2026-08-13 · Vayalet Stefanova, Diwas Lamsal, Margot Genbrugge, Maxim Yudayev 외 arxiv

Understanding motion in daily living requires context beyond kinematics, because similar inertial patterns during activities of daily living (ADLs) can reflect intentional stopping, object interaction, or pathological mo…

EgoTaskQA: Understanding Human Tasks in Egocentric Videos

2022-10-08 · Baoxiong Jia, Ting Lei, Song-Chun Zhu, Siyuan Huang

Understanding human tasks through video observations is an essential capability of intelligent agents. The challenges of such capability lie in the difficulty of generating a detailed understanding of situated actions, t…

Action LocalizationcounterfactualDescriptiveDiagnostic+3

Visual Intention Grounding for Egocentric Assistants

2025-04-18 · Pengzhan Sun, Junbin Xiao, Tze Ho Elden Tse, Yicong Li 외

Visual grounding associates textual descriptions with objects in an image. Conventional methods target third-person image inputs and named object queries. In applications such as AI assistants, the perspective shifts -- …

ObjectVisual Grounding