VideoSSR: Video Self-Supervised Reinforcement Learning

2025-11-09 · Zefeng He, Xiaoye Qu, Yafu Li, Siyuan Huang, Daizong Liu, Yu Cheng arxiv

Reinforcement Learning with Verifiable Rewards (RLVR) has substantially advanced the video understanding capabilities of Multimodal Large Language Models (MLLMs). However, the rapid progress of MLLMs is outpacing the complexity of existing video datasets, while the manual annotation of new, high-quality data remains prohibitively expensive. This work investigates a pivotal question: Can the rich, intrinsic information within videos be harnessed to self-generate high-quality, verifiable training data? To investigate this, we introduce three self-supervised pretext tasks: Anomaly Grounding, Object Counting, and Temporal Jigsaw. We construct the Video Intrinsic Understanding Benchmark (VIUBench) to validate their difficulty, revealing that current state-of-the-art MLLMs struggle significantly on these tasks. Building upon these pretext tasks, we develop the VideoSSR-30K dataset and propose VideoSSR, a novel video self-supervised reinforcement learning framework for RLVR. Extensive experiments across 17 benchmarks, spanning four major video domains (General Video QA, Long Video QA, Temporal Grounding, and Complex Reasoning), demonstrate that VideoSSR consistently enhances model performance, yielding an average improvement of over 5\%. These results establish VideoSSR as a potent foundational framework for developing more advanced video understanding in MLLMs. The code is available at https://github.com/lcqysl/VideoSSR.

📄 PDF Abstract BibTeX arXiv:2511.06281

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning Object Counting

Self-Attention Recurrent Summarization Network with Reinforcement Learning for Video Summarization Task

2021-06-09 · IEEE International Conference on Multimedia and Expo (ICME) 2021 6 · Aniwat Phaphuangwittayakul, Yi Guo, Fangli Ying, Wentian Xu 외

With the exponential growth of video data, video summarization techniques are urgently needed for reducing people’s efforts in the videos' content exploration by generating succinct but informative summaries from origina…

reinforcement-learning Reinforcement Learning Supervised Video Summarization Unsupervised Video Summarization+1

TRIMMER: A New Paradigm for Video Summarization through Self-Supervised Reinforcement Learning

2026-05-03 · Pritam Mishra, Coloma Ballester, Dimosthenis Karatzas arxiv

The rapid growth of video content across domains such as surveillance, education, and social media has made efficient content understanding increasingly critical. Video summarization addresses this challenge by generatin…

Self-Supervised Learning Computational Efficiency Reinforcement Learning Video Summarization

ViSS-R1: Self-Supervised Reinforcement Video Reasoning

2025-11-17 · Bo Fang, Yuxin Song, Qiangqiang Wu, Haoyuan Sun 외 arxiv

Complex video reasoning remains a significant challenge for Multimodal Large Language Models (MLLMs), as current R1-based methodologies often prioritize text-centric reasoning derived from text-based and image-based deve…

Self-Supervised Learning Reinforcement Learning

Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedback

2024-02-06 · Daechul Ahn, Yura Choi, Youngjae Yu, Dongyeop Kang 외

Recent advancements in large language models have influenced the development of video large multimodal models (VLMMs). The previous approaches for VLMMs involved Supervised Fine-Tuning (SFT) with instruction-tuned datase…

Video-based Generative Performance Benchmarking

Planning to Explore via Self-Supervised World Models

2020-05-12 · Ramanan Sekar, Oleh Rybkin, Kostas Daniilidis, Pieter Abbeel 외

Reinforcement learning allows solving complex tasks, however, the learning tends to be task-specific and the sample efficiency remains a challenge. We present Plan2Explore, a self-supervised reinforcement learning agent …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)