Model-based Reinforcement Learning with Parametrized Physical Models and Optimism-Driven Exploration

2015-09-23 · Christopher Xie, Sachin Patil, Teodor Moldovan, Sergey Levine, Pieter Abbeel

In this paper, we present a robotic model-based reinforcement learning method that combines ideas from model identification and model predictive control. We use a feature-based representation of the dynamics that allows the dynamics model to be fitted with a simple least squares procedure, and the features are identified from a high-level specification of the robot's morphology, consisting of the number and connectivity structure of its links. Model predictive control is then used to choose the actions under an optimistic model of the dynamics, which produces an efficient and goal-directed exploration strategy. We present real time experimental results on standard benchmark problems involving the pendulum, cartpole, and double pendulum systems. Experiments indicate that our method is able to learn a range of benchmark tasks substantially faster than the previous best methods. To evaluate our approach on a realistic robotic control task, we also demonstrate real time control of a simulated 7 degree of freedom arm.

📄 PDF Abstract BibTeX arXiv:1509.06824

Code (0)

등록된 구현이 없습니다.

Tasks

Model-based Reinforcement LearningModel Predictive Controlreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

QUOTA: The Quantile Option Architecture for Reinforcement Learning

2018-11-05 · Shangtong Zhang, Borislav Mavrin, Linglong Kong, Bo Liu 외

In this paper, we propose the Quantile Option Architecture (QUOTA) for exploration based on recent advances in distributional reinforcement learning (RL). In QUOTA, decision making is based on quantiles of a value distri…

Decision Making Distributional Reinforcement Learning reinforcement-learning Reinforcement Learning+1

OVD-Explorer: Optimism Should Not Be the Sole Pursuit of Exploration in Noisy Environments

2023-12-19 · Jinyi Liu, Zhi Wang, Yan Zheng, Jianye Hao 외

In reinforcement learning, the optimism in the face of uncertainty (OFU) is a mainstream principle for directing exploration towards less explored areas, characterized by higher uncertainty. However, in the presence of e…

continuous-control Continuous Control MuJoCo

Exploring Pessimism and Optimism Dynamics in Deep Reinforcement Learning

2024-06-06 · Bahareh Tasdighi, Nicklas Werge, Yi-Shan Wu, Melih Kandemir

Off-policy actor-critic algorithms have shown promise in deep reinforcement learning for continuous control tasks. Their success largely stems from leveraging pessimistic state-action value function updates, which effect…

continuous-control Continuous Control Deep Reinforcement Learning reinforcement-learning+1

Beyond Optimism: Exploration With Partially Observable Rewards

2024-06-20 · Simone Parisi, Alireza Kazemipour, Michael Bowling

Exploration in reinforcement learning (RL) remains an open challenge. RL algorithms rely on observing rewards to train the agent, and if informative rewards are sparse the agent learns slowly or may not learn at all. To …

Benchmarking Reinforcement Learning (RL)

Robust Exploration with Tight Bayesian Plausibility Sets

2019-04-17 · Reazul H. Russel, Tianyi Gu, Marek Petrik

Optimism about the poorly understood states and actions is the main driving force of exploration for many provably-efficient reinforcement learning algorithms. We propose optimism in the face of sensible value functions …

Reinforcement Learning