Pop-Up Distractions Reveal Bag-of-Events Behavior in Video Large Language Models
A key capability for video understanding is reliably linking subjects to events across time, yet whether Video Large Language Models (VideoLLMs) actually achieve this remains unclear. In this work, we introduce DistractionBench to evaluate whether VideoLLMs can robustly link subjects and events in the presence of unrelated video segments. Through controlled interventions, such as inserting short advertisement clips into longer videos, we show that VideoLLMs frequently hallucinate interactions between entities from different segments, incorrectly attributing actions from injected advertisements to subjects in the main video. We characterize this systematic hallucination as bag-of-events (BoE) behavior, where models process videos as collections of events rather than temporally structured sequences. Evaluating 11 popular VideoLLMs, we find that all models exhibit substantial BoE behavior. Our findings suggest that VideoLLMs lack reliable mechanisms for temporal grounding and motivate the development of models with more robust subject-event association.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Caution for the Environment: Multimodal Agents are Susceptible to Environmental Distractions
This paper investigates the faithfulness of multimodal large language model (MLLM) agents in the graphical user interface (GUI) environment, aiming to address the research question of whether multimodal GUI agents can be…
Language ModelingLanguage ModellingLarge Language ModelMultimodal Large Language ModelBiometrics and Behavior Analysis for Detecting Distractions in e-Learning
In this article, we explore computer vision approaches to detect abnormal head pose during e-learning sessions and we introduce a study on the effects of mobile phone usage during these sessions. We utilize behavioral da…
Human-Centric Anomaly Detection in Surveillance Videos Using YOLO-World and Spatio-Temporal Deep Learning
Anomaly detection in surveillance videos remains a challenging task due to the diversity of abnormal events, class imbalance, and scene-dependent visual clutter. To address these issues, we propose a robust deep learning…
Anomaly Detection In Surveillance VideosAnomaly ClassificationChaotic World: A Large and Challenging Benchmark for Human Behavior Understanding in Chaotic Events
Understanding and analyzing human behaviors (actions and interactions of people), voices, and sounds in chaotic events is crucial in many applications, e.g., crowd management, emergency response services. Different f…
Action LocalizationPathfinderSound Source LocalizationBehavioral Research and Practical Models of Drivers' Attention
Driving is a routine activity for many, but it is far from simple. Drivers deal with multiple concurrent tasks, such as keeping the vehicle in the lane, observing and anticipating the actions of other road users, reactin…