HiProbe-VAD: Video Anomaly Detection via Hidden States Probing in Tuning-Free Multimodal LLMs
Video Anomaly Detection (VAD) aims to identify and locate deviations from normal patterns in video sequences. Traditional methods often struggle with substantial computational demands and a reliance on extensive labeled datasets, thereby restricting their practical applicability. To address these constraints, we propose HiProbe-VAD, a novel framework that leverages pre-trained Multimodal Large Language Models (MLLMs) for VAD without requiring fine-tuning. In this paper, we discover that the intermediate hidden states of MLLMs contain information-rich representations, exhibiting higher sensitivity and linear separability for anomalies compared to the output layer. To capitalize on this, we propose a Dynamic Layer Saliency Probing (DLSP) mechanism that intelligently identifies and extracts the most informative hidden states from the optimal intermediate layer during the MLLMs reasoning. Then a lightweight anomaly scorer and temporal localization module efficiently detects anomalies using these extracted hidden states and finally generate explanations. Experiments on the UCF-Crime and XD-Violence datasets demonstrate that HiProbe-VAD outperforms existing training-free and most traditional approaches. Furthermore, our framework exhibits remarkable cross-model generalization capabilities in different MLLMs without any tuning, unlocking the potential of pre-trained MLLMs for video anomaly detection and paving the way for more practical and scalable solutions.
Code (0)
등록된 구현이 없습니다.
Tasks
Video Anomaly DetectionSimilar Papers 제목 키워드 기반
Bayesian Nonparametric Submodular Video Partition for Robust Anomaly Detection
Multiple-instance learning (MIL) provides an effective way to tackle the video anomaly detection problem by modeling it as a weakly supervised problem as the labels are usually only available at the video level while mis…
Anomaly DetectionMultiple Instance LearningVideo Anomaly DetectionWeakly-supervised Video Anomaly DetectionSearching for a Hidden Markov Anomaly over Multiple Processes
We address the problem of detecting an anomalous process among a large number of processes. At each time t, normal processes are in state zero (normal state), while the abnormal process may be in either state zero (norma…
Anomaly DetectionA MIL Approach for Anomaly Detection in Surveillance Videos from Multiple Camera Views
Occlusion and clutter are two scene states that make it difficult to detect anomalies in surveillance video. Furthermore, anomaly events are rare and, as a consequence, class imbalance and lack of labeled anomaly data ar…
Anomaly DetectionAnomaly Detection In Surveillance VideosMultiple Instance LearningRoot Cause Detection Among Anomalous Time Series Using Temporal State Alignment
The recent increase in the scale and complexity of software systems has introduced new challenges to the time series monitoring and anomaly detection process. A major drawback of existing anomaly detection methods is tha…
Anomaly DetectionTime SeriesTime Series AnalysisAdaptive State-Space Mamba for Real-Time Sensor Data Anomaly Detection
State-space modeling has emerged as a powerful paradigm for sequence analysis in various tasks such as natural language processing, time-series forecasting, and signal processing. In this work, we propose an \emph{Adapti…
Anomaly DetectionMambaState Space ModelsStyle Transfer+2