Evaluating task-agnostic exploration for fixed-batch learning of arbitrary future tasks
Deep reinforcement learning has been shown to solve challenging tasks where large amounts of training experience is available, usually obtained online while learning the task. Robotics is a significant potential application domain for many of these algorithms, but generating robot experience in the real world is expensive, especially when each task requires a lengthy online training procedure. Off-policy algorithms can in principle learn arbitrary tasks from a diverse enough fixed dataset. In this work, we evaluate popular exploration methods by generating robotics datasets for the purpose of learning to solve tasks completely offline without any further interaction in the real world. We present results on three popular continuous control tasks in simulation, as well as continuous control of a high-dimensional real robot arm. Code documenting all algorithms, experiments, and hyper-parameters is available at https://github.com/qutrobotlearning/batchlearning.
Code (1)
Tasks
continuous-controlContinuous ControlDeep Reinforcement LearningReinforcement LearningSimilar Papers 제목 키워드 기반
Batch Exploration with Examples for Scalable Robotic Reinforcement Learning
Learning from diverse offline datasets is a promising path towards learning general purpose robotic agents. However, a core challenge in this paradigm lies in collecting large amounts of meaningful data, while not depend…
Offline RLreinforcement-learningReinforcement LearningReinforcement Learning (RL)Interesting Object, Curious Agent: Learning Task-Agnostic Exploration
Common approaches for task-agnostic exploration learn tabula-rasa --the agent assumes isolated environments and no prior knowledge or experience. However, in the real world, agents learn in many environments and always c…
ObjectOff-Policy Deep Reinforcement Learning without Exploration
Many practical applications of reinforcement learning constrain agents to learn from a fixed batch of data which has already been gathered, without offering further possibility for data collection. In this paper, we demo…
continuous-controlContinuous ControlDeep Reinforcement Learningreinforcement-learning+2A Batch Sequential Halving Algorithm without Performance Degradation
In this paper, we investigate the problem of pure exploration in the context of multi-armed bandits, with a specific focus on scenarios where arms are pulled in fixed-size batches. Batching has been shown to enhance comp…
Computational EfficiencyMulti-Armed BanditsParallel Gaussian Process Optimization with Upper Confidence Bound and Pure Exploration
In this paper, we consider the challenge of maximizing an unknown function f for which evaluations are noisy and are acquired with high cost. An iterative procedure uses the previous measures to actively select the next …