IRASim: Learning Interactive Real-Robot Action Simulators
Scalable robot learning in the real world is limited by the cost and safety issues of real robots. In addition, rolling out robot trajectories in the real world can be time-consuming and labor-intensive. In this paper, we propose to learn an interactive real-robot action simulator as an alternative. We introduce a novel method, IRASim, which leverages the power of generative models to generate extremely realistic videos of a robot arm that executes a given action trajectory, starting from an initial given frame. To validate the effectiveness of our method, we create a new benchmark, IRASim Benchmark, based on three real-robot datasets and perform extensive experiments on the benchmark. Results show that IRASim outperforms all the baseline methods and is more preferable in human evaluations. We hope that IRASim can serve as an effective and scalable approach to enhance robot learning in the real world. To promote research for generative real-robot action simulators, we open-source code, benchmark, and checkpoints at https: //gen-irasim.github.io.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
LOTUSim: Multi-Domain Simulator for Marine Robotics
Simulation is essential for maritime robotics, supporting operator training, mission rehearsal, and human-vehicle interaction in environments where real-world testing is costly or hazardous. Existing simulators focus pri…
Robot Air Hockey: A Manipulation Testbed for Robot Learning with Reinforcement Learning
Reinforcement Learning is a promising tool for learning complex policies even in fast-moving and object-interactive domains where human teleoperation or hard-coded policies might fail. To effectively reflect this challen…
Offline RLVR-Robo: A Real-to-Sim-to-Real Framework for Visual Robot Navigation and Locomotion
Recent success in legged robot locomotion is attributed to the integration of reinforcement learning and physical simulators. However, these policies often encounter challenges when deployed in real-world environments du…
3DGSreinforcement-learningReinforcement LearningRobot Navigation+1Human-in-the-loop Embodied Intelligence with Interactive Simulation Environment for Surgical Robot Learning
Surgical robot automation has attracted increasing research interest over the past decade, expecting its potential to benefit surgeons, nurses and patients. Recently, the learning paradigm of embodied intelligence has de…
RoDyn: Taming Interactive Robot-Dynamic 2.5D World Model for Robotic Manipulation
Learned world models hold significant potential as neural simulators for robotic manipulation. However, prevalent 2D video-based models inherently lack the spatial and kinematic reasoning crucial for physical interaction…
Reinforcement Learning