Self-Supervised Continuous Control without Policy Gradient
Despite the remarkable progress made by the policy gradient algorithms in reinforcement learning (RL), sub-optimal policies usually result from the local exploration property of the policy gradient update. In this work, we propose a method called Zeroth-Order Supervised Policy Improvement (ZOSPI) that exploits the estimated value function $Q$ globally while preserving the local exploitation of the policy gradient methods. Experiments show that ZOSPI achieves competitive results on the MuJoCo benchmarks with a remarkable sample efficiency. Moreover, different from the conventional policy gradient methods, the policy learning of ZOSPI is conducted in a self-supervised manner. We show such a self-supervised learning paradigm has the flexibility of including optimistic exploration as well as adopting a non-parametric policy.
Code (0)
등록된 구현이 없습니다.
Tasks
continuous-controlContinuous ControlMuJoCoPolicy Gradient Methodsreinforcement-learningReinforcement Learning (RL)Self-Supervised LearningSimilar Papers 제목 키워드 기반
Cross-domain Random Pre-training with Prototypes for Reinforcement Learning
This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. Unsupervised cross-domain Reinforcement Learning (RL) …
reinforcement-learningReinforcement Learning (RL)Representation LearningSelf-Supervised On-Policy Reinforcement Learning via Contrastive Proximal Policy Optimisation
Contrastive reinforcement learning (CRL) learns goal-conditioned Q-values through a contrastive objective over state-action and goal representations, removing the need for hand-crafted reward functions. Despite impressiv…
Self-Supervised LearningReinforcement LearningSelf-Supervised Policy Adaptation during Deployment
In most real world scenarios, a policy trained by reinforcement learning in one environment needs to be deployed in another, potentially quite different environment. However, generalization across different environments …
Supervised Learning for Stochastic Optimal Control
Supervised machine learning is powerful. In recent years, it has enabled massive breakthroughs in computer vision and natural language processing. But leveraging these advances for optimal control has proved difficult. D…
Policy Consolidation for Continual Reinforcement Learning
We propose a method for tackling catastrophic forgetting in deep reinforcement learning that is \textit{agnostic} to the timescale of changes in the distribution of experiences, does not require knowledge of task boundar…
Continual Learningcontinuous-controlContinuous ControlDeep Reinforcement Learning+3