Intervention Design for Effective Sim2Real Transfer
The goal of this work is to address the recent success of domain randomization and data augmentation for the sim2real setting. We explain this success through the lens of causal inference, positioning domain randomization and data augmentation as interventions on the environment which encourage invariance to irrelevant features. Such interventions include visual perturbations that have no effect on reward and dynamics. This encourages the learning algorithm to be robust to these types of variations and learn to attend to the true causal mechanisms for solving the task. This connection leads to two key findings: (1) perturbations to the environment do not have to be realistic, but merely show variation along dimensions that also vary in the real world, and (2) use of an explicit invariance-inducing objective improves generalization in sim2sim and sim2real transfer settings over just data augmentation or domain randomization alone. We demonstrate the capability of our method by performing zero-shot transfer of a robot arm reach task on a 7DoF Jaco arm learning from pixel observations.
Code (1)
Tasks
Causal InferenceData AugmentationSimilar Papers 제목 키워드 기반
Designing and evaluating an online reinforcement learning agent for physical exercise recommendations in N-of-1 trials
Personalized adaptive interventions offer the opportunity to increase patient benefits, however, there are challenges in their planning and implementation. Once implemented, it is an important question whether personaliz…
reinforcement-learningReinforcement LearningDiscovering User Types: Mapping User Traits by Task-Specific Behaviors in Reinforcement Learning
When assisting human users in reinforcement learning (RL), we can represent users as RL agents and study key parameters, called \emph{user traits}, to inform intervention design. We study the relationship between user be…
reinforcement-learningReinforcement Learning (RL)Out of Distribution Generalization via Interventional Style Transfer in Single-Cell Microscopy
Real-world deployment of computer vision systems, including in the discovery processes of biomedical research, requires causal representations that are invariant to contextual nuisances and generalize to new data. Levera…
Out-of-Distribution GeneralizationStyle TransferAnyBipe: An End-to-End Framework for Training and Deploying Bipedal Robots Guided by Large Language Models
Training and deploying reinforcement learning (RL) policies for robots, especially in accomplishing specific tasks, presents substantial challenges. Recent advancements have explored diverse reward function designs, trai…
Reinforcement Learning (RL)SlicerROS2: A Research and Development Module for Image-Guided Robotic Interventions
Image-guided robotic interventions involve the use of medical imaging in tandem with robotics. SlicerROS2 is a software module that combines 3D Slicer and robot operating system (ROS) in pursuit of a standard integration…