Measuring Visual Generalization in Continuous Control from Pixels
Self-supervised learning and data augmentation have significantly reduced the performance gap between state and image-based reinforcement learning agents in continuous control tasks. However, it is still unclear whether current techniques can face a variety of visual conditions required by real-world environments. We propose a challenging benchmark that tests agents' visual generalization by adding graphical variety to existing continuous control domains. Our empirical analysis shows that current methods struggle to generalize across a diverse set of visual changes, and we examine the specific factors of variation that make these tasks difficult. We find that data augmentation techniques outperform self-supervised learning approaches and that more significant image transformations provide better visual generalization \footnote{The benchmark and our augmented actor-critic implementation are open-sourced @ https://github.com/QData/dmc_remastered)
Code (2)
Tasks
continuous-controlContinuous ControlData AugmentationReinforcement Learning (RL)Self-Supervised LearningSimilar Papers 제목 키워드 기반
Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learning
Deep reinforcement learning policies, despite their outstanding efficiency in simulated visual control tasks, have shown disappointing ability to generalize across disturbances in the input training images. Changes in im…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learning
One of the key challenges in visual Reinforcement Learning (RL) is to learn policies that can generalize to unseen environments. Recently, data augmentation techniques aiming at enhancing data diversity have demonstrated…
Data AugmentationDiversityReinforcement Learning (RL)Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning
We present DrQ-v2, a model-free reinforcement learning (RL) algorithm for visual continuous control. DrQ-v2 builds on DrQ, an off-policy actor-critic approach that uses data augmentation to learn directly from pixels. We…
continuous-controlContinuous ControlData AugmentationGPU+4Sim-to-real reinforcement learning applied to end-to-end vehicle control
In this work, we study vision-based end-to-end reinforcement learning on vehicle control problems, such as lane following and collision avoidance. Our controller policy is able to control a small-scale robot to follow th…
Collision Avoidancereinforcement-learningReinforcement Learning (RL)Transfer LearningPotential Contrast: Properties, Equivalences, and Generalization to Multiple Classes
Potential contrast is typically used as an image quality measure and quantifies the maximal possible contrast between samples from two classes of pixels in an image after an arbitrary grayscale transformation. It has bee…