AdaptSim: Task-Driven Simulation Adaptation for Sim-to-Real Transfer
Simulation parameter settings such as contact models and object geometry approximations are critical to training robust robotic policies capable of transferring from simulation to real-world deployment. Previous approaches typically handcraft distributions over such parameters (domain randomization), or identify parameters that best match the dynamics of the real environment (system identification). However, there is often an irreducible gap between simulation and reality: attempting to match the dynamics between simulation and reality across all states and tasks may be infeasible and may not lead to policies that perform well in reality for a specific task. Addressing this issue, we propose AdaptSim, a new task-driven adaptation framework for sim-to-real transfer that aims to optimize task performance in target (real) environments -- instead of matching dynamics between simulation and reality. First, we meta-learn an adaptation policy in simulation using reinforcement learning for adjusting the simulation parameter distribution based on the current policy's performance in a target environment. We then perform iterative real-world adaptation by inferring new simulation parameter distributions for policy training, using a small amount of real data. We perform experiments in three robotic tasks: (1) swing-up of linearized double pendulum, (2) dynamic table-top pushing of a bottle, and (3) dynamic scooping of food pieces with a spatula. Our extensive simulation and hardware experiments demonstrate AdaptSim achieving 1-3x asymptotic performance and $\sim$2x real data efficiency when adapting to different environments, compared to methods based on Sys-ID and directly training the task policy in target environments. Website: https://irom-lab.github.io/AdaptSim/
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
CTS: Sim-to-Real Unsupervised Domain Adaptation on 3D Detection
Simulation data can be accurately labeled and have been expected to improve the performance of data-driven algorithms, including object detection. However, due to the various domain inconsistencies from simulation to rea…
3D Object DetectionDomain AdaptationObjectobject-detection+3Sym2Real: Symbolic Dynamics with Residual Learning for Data-Efficient Adaptive Control
We present Sym2Real, a fully data-driven framework for highly data-efficient adaptation of low-level controllers. Although symbolic regression is data-efficient, its role in real-world control has been limited due to its…
What Robot do I Need? Fast Co-Adaptation of Morphology and Control using Graph Neural Networks
The co-adaptation of robot morphology and behaviour becomes increasingly important with the advent of fast 3D-manufacturing methods and efficient deep reinforcement learning algorithms. A major challenge for the applicat…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)QTIP: Quick simulation-based adaptation of Traffic model per Incident Parameters
Current data-driven traffic prediction models are usually trained with large datasets, e.g. several months of speeds and flows. Such models provide very good fit for ordinary road conditions, but often fail just when the…
PredictionTraffic PredictionSim-Anchored Learning for On-the-Fly Adaptation
Fine-tuning simulation-trained RL agents with real-world data often degrades crucial behaviors due to limited or skewed data distributions. We argue that designer priorities exist not just in reward functions, but also i…
Reinforcement Learning (RL)