Equivariant Data Augmentation for Generalization in Offline Reinforcement Learning
We present a novel approach to address the challenge of generalization in offline reinforcement learning (RL), where the agent learns from a fixed dataset without any additional interaction with the environment. Specifically, we aim to improve the agent's ability to generalize to out-of-distribution goals. To achieve this, we propose to learn a dynamics model and check if it is equivariant with respect to a fixed type of transformation, namely translations in the state space. We then use an entropy regularizer to increase the equivariant set and augment the dataset with the resulting transformed samples. Finally, we learn a new policy offline based on the augmented dataset, with an off-the-shelf offline RL algorithm. Our experimental results demonstrate that our approach can greatly improve the test performance of the policy on the considered environments.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationOffline RLreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Equivariant Offline Reinforcement Learning
Sample efficiency is critical when applying learning-based methods to robotic manipulation due to the high cost of collecting expert demonstrations and the challenges of on-robot policy learning through online Reinforcem…
Offline RLQ-Learningreinforcement-learningReinforcement Learning+1Data Augmentation vs. Equivariant Networks: A Theory of Generalization on Dynamics Forecasting
Exploiting symmetry in dynamical systems is a powerful way to improve the generalization of deep learning. The model learns to be invariant to transformation and hence is more robust to distribution shift. Data augmentat…
Data AugmentationGeneralization BoundsOffline Trajectory Generalization for Offline Reinforcement Learning
Offline reinforcement learning (RL) aims to learn policies from static datasets of previously collected trajectories. Existing methods for offline RL either constrain the learned policy to the support of offline data or …
D4RLData AugmentationOffline RLreinforcement-learning+2Equivariant Goal Conditioned Contrastive Reinforcement Learning
Contrastive Reinforcement Learning (CRL) provides a promising framework for extracting useful structured representations from unlabeled interactions. By pulling together state-action pairs and their corresponding future …
Reinforcement LearningOffline RLMulti-Group Equivariant Augmentation for Reinforcement Learning in Robot Manipulation
Sampling efficiency is critical for deploying visuomotor learning in real-world robotic manipulation. While task symmetry has emerged as a promising inductive bias to improve efficiency, most prior work is limited to iso…
Reinforcement LearningRobot ManipulationData Augmentation