World-Gymnast: Training Robots with Reinforcement Learning in a World Model

2026-02-02 · Ansh Kumar Sharma, Yixiang Sun, Ninghao Lu, Yunzhe Zhang, Jiarao Liu, Sherry Yang arxiv

Robot learning from interacting with the physical world is fundamentally bottlenecked by the cost of physical interaction. The two alternatives, supervised finetuning (SFT) from expert demonstrations and reinforcement learning (RL) in a software-based simulator, are limited by the amount of expert data available and the sim-to-real gap for manipulation. With the recent emergence of world models learned from real-world video-action data, we ask the question of whether training a policy in a world model can be more effective than supervised learning or software simulation in achieving better real-robot performance. We propose World-Gymnast, which performs RL finetuning of a vision-language-action (VLA) policy by rolling out the policy in an action-conditioned video world model and rewarding the rollouts with a vision-language model (VLM). On the Bridge robot setup, World-Gymnast outperforms SFT by as much as 18x and outperforms software simulator by as much as 2x. More importantly, World-Gymnast demonstrates intriguing capabilities of RL with a world model, including training on diverse language instructions and novel scenes from the world model, test-time training in a novel scene, and online iterative world model and policy improvement. Our results suggest learning a world model and training robot policies in the cloud could be the key to bridging the gap between robots that work in demonstrations and robots that can work in anyone's household.

📄 PDF Abstract BibTeX arXiv:2602.02454

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Enhancing Multi-Camera Gymnast Tracking Through Domain Knowledge Integration

2025-11-20 · Fan Yang, Shigeyuki Odashima, Shoichi Masui, Ikuo Kusajima 외 arxiv

We present a robust multi-camera gymnast tracking, which has been applied at international gymnastics championships for gymnastics judging. Despite considerable progress in multi-camera tracking algorithms, tracking gymn…

Collective Robot Reinforcement Learning with Distributed Asynchronous Guided Policy Search

2016-10-03 · Ali Yahya, Adrian Li, Mrinal Kalakrishnan, Yevgen Chebotar 외

In principle, reinforcement learning and policy search methods can enable robots to learn highly complex and general skills that may allow them to function amid the complexity and diversity of the real world. However, tr…

Diversity reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Benchmarking Reinforcement Learning Algorithms on Real-World Robots

2018-09-20 · A. Rupam Mahmood, Dmytro Korenkevych, Gautham Vasan, William Ma 외

Through many recent successes in simulation, model-free reinforcement learning has emerged as a promising approach to solving continuous control robotic tasks. The research community is now able to reproduce, analyze and…

Benchmarking continuous-control Continuous Control reinforcement-learning+2

Setting up a Reinforcement Learning Task with a Real-World Robot

2018-03-19 · A. Rupam Mahmood, Dmytro Korenkevych, Brent J. Komer, James Bergstra

Reinforcement learning is a promising approach to developing hard-to-engineer adaptive solutions for complex and diverse robotic tasks. However, learning with real-world robots is often unreliable and difficult, which re…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

robo-gym -- An Open Source Toolkit for Distributed Deep Reinforcement Learning on Real and Simulated Robots

2020-07-06 · Matteo Lucchi, Friedemann Zindler, Stephan Mühlbacher-Karrer, Horst Pichler

Applying Deep Reinforcement Learning (DRL) to complex tasks in the field of robotics has proven to be very successful in the recent years. However, most of the publications focus either on applying it to a task in simula…

Deep Reinforcement Learning Industrial Robots Transfer Learning