홈 › Papers

ViVa: A Video-Generative Value Model for Robot Reinforcement Learning

2026-04-09 · Jindi Lv, Hao Li, Jie Li, Fankun Kong, Yang Wang, Pengfei Yi, Yifei Nie, Xiaofeng Wang, Zheng Zhu, Chaojun Ni, Qiuping Deng, Hengtao Li, Jiancheng Lv, Guan Huang arxiv

Vision-language-action (VLA) models have advanced robot manipulation through large-scale pretraining, but real-world deployment remains challenging due to partial observability and delayed feedback. Reinforcement learning addresses this via value functions, which assess task progress and guide policy improvement. However, existing value models built on vision-language models (VLMs) struggle to capture temporal dynamics and physical interactions, undermining reliable value estimation in long-horizon tasks. In this paper, we propose ViVa, a video-generative value model that repurposes a pretrained video generator to jointly predict future proprioception and a scalar value. By grounding value estimation in anticipated embodiment dynamics, ViVa leverages spatiotemporal priors to intrinsically couple value with foresight beyond static snapshots. ViVa achieves state-of-the-art results in metric-based evaluation across three tasks, producing reliable value signals that accurately track task progress and detect execution errors. Integrated into RECAP, it achieves an average success rate of 80%, highlighting the promise of video-generative models for value estimation.

📄 PDF Abstract BibTeX arXiv:2604.08168

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning Robot Manipulation

SVL: Goal-Conditioned Reinforcement Learning as Survival Learning

2026-04-19 · Franki Nguimatsia Tiofack, Fabian Schramm, Théotime Le Hellard, Justin Carpentier arxiv

Standard approaches to goal-conditioned reinforcement learning (GCRL) that rely on temporal-difference learning can be unstable and sample-inefficient due to bootstrapping. While recent work has explored contrastive and …

Reinforcement Learning

Survival Reinforcement Learning: Toward Scalable Self-Supervised RL

2026-05-29 · Franki Nguimatsia-Tiofack, Fabian Schramm, Théotime Le Hellard, Justin Carpentier arxiv

While self-supervised Contrastive Reinforcement Learning (CRL) has shown remarkable depth-scaling capabilities, successfully using networks over 64 layers, scaled CRL still struggles with long-horizon goal-conditioned pl…

Reinforcement Learning

Robotic Offline RL from Internet Videos via Value-Function Pre-Training

2023-09-22 · Chethan Bhateja, Derek Guo, Dibya Ghosh, Anikait Singh 외

Pre-training on Internet data has proven to be a key ingredient for broad generalization in many modern ML systems. What would it take to enable such capabilities in robotic reinforcement learning (RL)? Offline RL method…

Offline RL Reinforcement Learning (RL)

Reinforcement Learning with Videos: Combining Offline Observations with Interaction

2020-11-12 · Karl Schmeckpeper, Oleh Rybkin, Kostas Daniilidis, Sergey Levine 외

Reinforcement learning is a powerful framework for robots to acquire skills from experience, but often requires a substantial amount of online data collection. As a result, it is difficult to collect sufficiently diverse…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Generative Multi-Robot Motion Planning via Diffusion Modeling with Multi-Agent Reinforcement Learning Guidance

2026-05-30 · Suk Ki Lee, Venkata Sai Deepak Mutta, Hyunwoong Ko arxiv

Coordinating multiple robots in shared environments requires generating feasible trajectories for each agent while accounting for interactions among agents. Centralized planning approaches become difficult to scale as th…

Multi-agent Reinforcement Learning Trajectory Planning Motion Planning