Graph Inverse Reinforcement Learning from Diverse Videos
Research on Inverse Reinforcement Learning (IRL) from third-person videos has shown encouraging results on removing the need for manual reward design for robotic tasks. However, most prior works are still limited by training from a relatively restricted domain of videos. In this paper, we argue that the true potential of third-person IRL lies in increasing the diversity of videos for better scaling. To learn a reward function from diverse videos, we propose to perform graph abstraction on the videos followed by temporal matching in the graph space to measure the task progress. Our insight is that a task can be described by entity interactions that form a graph, and this graph abstraction can help remove irrelevant information such as textures, resulting in more robust reward functions. We evaluate our approach, GraphIRL, on cross-embodiment learning in X-MAGICAL and learning from human demonstrations for real-robot manipulation. We show significant improvements in robustness to diverse video demonstrations over previous approaches, and even achieve better results than manual reward design on a real robot pushing task. Videos are available at https://sateeshkumar21.github.io/GraphIRL .
Code (0)
등록된 구현이 없습니다.
Tasks
Diversityreinforcement-learningReinforcement LearningReinforcement Learning (RL)Robot ManipulationSimilar Papers 제목 키워드 기반
Diffusion Renderer: Neural Inverse and Forward Rendering with Video Diffusion Models
Understanding and modeling lighting effects are fundamental tasks in computer vision and graphics. Classic physically-based rendering (PBR) accurately simulates the light transport, but relies on precise scene repres…
3D geometryInverse RenderingReinforcement Learning with Inverse Rewards for World Model Post-training
World models simulate dynamic environments, enabling agents to interact with diverse input modalities. Although recent advances have improved the visual quality and temporal consistency of video world models, their abili…
Reinforcement LearningLearning Navigation Subroutines from Egocentric Videos
Planning at a higher level of abstraction instead of low level torques improves the sample efficiency in reinforcement learning, and computational efficiency in classical planning. We propose a method to learn such hiera…
Computational EfficiencyPseudo LabelReinforcement LearningA proof of imitation of Wasserstein inverse reinforcement learning for multi-objective optimization
We prove Wasserstein inverse reinforcement learning enables the learner's reward values to imitate the expert's reward values in a finite iteration for multi-objective optimizations. Moreover, we prove Wasserstein invers…
reinforcement-learningReinforcement LearningXIRL: Cross-embodiment Inverse Reinforcement Learning
We investigate the visual cross-embodiment imitation setting, in which agents learn policies from videos of other agents (such as humans) demonstrating the same task, but with stark differences in their embodiments -- sh…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)