Video-CoE: Reinforcing Video Event Prediction via Chain of Events
Despite advances in the application of MLLMs for various video tasks, video event prediction (VEP) remains relatively underexplored. VEP requires the model to perform fine-grained temporal modeling of videos and establish logical relationships between videos and future events, which current MLLMs still struggle with. In this work, we first present a comprehensive evaluation of current leading MLLMs on the VEP task, revealing the reasons behind their inaccurate predictions, including lack of logical reasoning ability for future events prediction and insufficient utilization of visual information. To address these challenges, we propose \textbf{C}hain \textbf{o}f \textbf{E}vents (\textbf{CoE}) paradigm, which constructs temporal event chains to implicitly enforce MLLM focusing on the visual content and the logical connections between videos and future events, incentivizing model's reasoning capability with multiple training protocols. Experimental results on public benchmarks demonstrate that our method outperforms both leading open-source and commercial MLLMs, establishing a new state-of-the-art on the VEP task. Codes and models will be released soon.
Code (0)
등록된 구현이 없습니다.
Tasks
Logical ReasoningSimilar Papers 제목 키워드 기반
Towards Effective Long-Video Event Prediction via Multi-Level Event Semantics Mining
Accurately predicting future events is fundamental to content understanding and decision-making across various domains. While prior research has primarily focused on text or short-video scenarios, long-video event predic…
Video Question AnsweringMVP: Enhancing Video Large Language Models via Self-supervised Masked Video Prediction
Reinforcement learning based post-training paradigms for Video Large Language Models (VideoLLMs) have achieved significant success by optimizing for visual-semantic tasks such as captioning or VideoQA. However, while the…
Reinforcement LearningVideo PredictionMECD+: Unlocking Event-Level Causal Graph Discovery for Video Reasoning
Video causal reasoning aims to achieve a high-level understanding of videos from a causal perspective. However, it exhibits limitations in its scope, primarily executed in a question-answering paradigm and focusing on br…
Causal DiscoveryCausal InferencecounterfactualCounterfactual Inference+3Reinforcing Video Reasoning Segmentation to Think Before It Segments
Video reasoning segmentation (VRS) endeavors to delineate referred objects in videos guided by implicit instructions that encapsulate human intent and temporal logic. Previous approaches leverage large vision language mo…
Reinforcement LearningGraphThinker: Reinforcing Temporally Grounded Video Reasoning with Event Graph Thinking
Video reasoning requires a fine-grained understanding of the temporal dependencies and event-level relations between objects and events in videos. Current Multimodal Large Language Models (MLLMs) are prone to severe temp…
Visual Grounding