Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning

2023-05-29 · NeurIPS 2023 11 · Jialong Wu, Haoyu Ma, Chaoyi Deng, Mingsheng Long

Unsupervised pre-training methods utilizing large and diverse datasets have achieved tremendous success across a range of domains. Recent work has investigated such unsupervised pre-training methods for model-based reinforcement learning (MBRL) but is limited to domain-specific or simulated data. In this paper, we study the problem of pre-training world models with abundant in-the-wild videos for efficient learning of downstream visual control tasks. However, in-the-wild videos are complicated with various contextual factors, such as intricate backgrounds and textured appearance, which precludes a world model from extracting shared world knowledge to generalize better. To tackle this issue, we introduce Contextualized World Models (ContextWM) that explicitly separate context and dynamics modeling to overcome the complexity and diversity of in-the-wild videos and facilitate knowledge transfer between distinct scenes. Specifically, a contextualized extension of the latent dynamics model is elaborately realized by incorporating a context encoder to retain contextual information and empower the image decoder, which encourages the latent dynamics model to concentrate on essential temporal variations. Our experiments show that in-the-wild video pre-training equipped with ContextWM can significantly improve the sample efficiency of MBRL in various domains, including robotic manipulation, locomotion, and autonomous driving. Code is available at this repository: https://github.com/thuml/ContextWM.

📄 PDF Abstract BibTeX arXiv:2305.18499

Code (1)

thuml/ContextWM 공식 구현 pytorch

Tasks

Autonomous Driving DecoderModel-based Reinforcement LearningTransfer Learning Unsupervised Pre-trainingWorld Knowledge

Human-to-Robot Imitation in the Wild

2022-07-19 · Shikhar Bahl, Abhinav Gupta, Deepak Pathak

We approach the problem of learning by watching humans in the wild. While traditional approaches in Imitation and Reinforcement Learning are promising for learning in the real world, they are either sample inefficient or…

Reinforcement Learning from Wild Animal Videos

2024-12-05 · Elliot Chane-Sane, Constant Roux, Olivier Stasse, Nicolas Mansard

We propose to learn legged robot locomotion skills by watching thousands of wild animal videos from the internet, such as those featured in nature documentaries. Indeed, such videos offer a rich and diverse collection of…

reinforcement-learning Reinforcement Learning

NTIRE 2026 Challenge on Short-form UGC Video Restoration in the Wild with Generative Models: Datasets, Methods and Results

2026-04-12 · Xin Li, Jiachao Gong, Xijun Wang, Shiyao Xiong 외 arxiv

This paper presents an overview of the NTIRE 2026 Challenge on Short-form UGC Video Restoration in the Wild with Generative Models. This challenge utilizes a new short-form UGC (S-UGC) video restoration benchmark, termed…

Video Restoration

Game2World Engine: Unlocking In-the-Wild Gameplay Videos for World Model Training

2026-08-25 · Wenxuan Shen, Dongna Jin, Dongping Chen hf

Video games provide a scalable source of training data for video world models, offering diverse environments, complex interactions, and abundant in-the-wild gameplay videos. However, raw gameplay footage entangles the ga…

Pre-trained Visual Dynamics Representations for Efficient Policy Learning

2024-11-05 · Hao Luo, Bohan Zhou, Zongqing Lu

Pre-training for Reinforcement Learning (RL) with purely video data is a valuable yet challenging problem. Although in-the-wild videos are readily available and inhere a vast amount of prior world knowledge, the absence …

Reinforcement Learning (RL)Video Prediction World Knowledge