ACE: An Actor Ensemble Algorithm for Continuous Control with Tree Search
In this paper, we propose an actor ensemble algorithm, named ACE, for continuous control with a deterministic policy in reinforcement learning. In ACE, we use actor ensemble (i.e., multiple actors) to search the global maxima of the critic. Besides the ensemble perspective, we also formulate ACE in the option framework by extending the option-critic architecture with deterministic intra-option policies, revealing a relationship between ensemble and options. Furthermore, we perform a look-ahead tree search with those actors and a learned value prediction model, resulting in a refined value estimation. We demonstrate a significant performance boost of ACE over DDPG and its variants in challenging physical robot simulators.
Code (1)
Tasks
continuous-controlContinuous Controlreinforcement-learningReinforcement LearningReinforcement Learning (RL)Value predictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
ControlBurn: Nonlinear Feature Selection with Sparse Tree Ensembles
ControlBurn is a Python package to construct feature-sparse tree ensembles that support nonlinear feature selection and interpretable machine learning. The algorithms in this package first build large tree ensembles that…
Additive modelsfeature selectionInterpretable Machine LearningEnsemble Reinforcement Learning in Continuous Spaces -- A Hierarchical Multi-Step Approach for Policy Training
Actor-critic deep reinforcement learning (DRL) algorithms have recently achieved prominent success in tackling various challenging reinforcement learning (RL) problems, particularly complex control tasks with high-dimens…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Tree-Averaging Algorithms for Ensemble-Based Unsupervised Discontinuous Constituency Parsing
We address unsupervised discontinuous constituency parsing, where we observe a high variance in the performance of the only previous model in the literature. We propose to build an ensemble of different runs of the exist…
AllConstituency ParsingDeep Exploration with PAC-Bayes
Reinforcement learning for continuous control under delayed rewards is an under-explored problem despite its significance in real life. Many complex skills build on intermediate ones as prerequisites. For instance, a hum…
continuous-controlContinuous ControlDecision Makingreinforcement-learning+1TASAC: a twin-actor reinforcement learning framework with stochastic policy for batch process control
Due to their complex nonlinear dynamics and batch-to-batch variability, batch processes pose a challenge for process control. Due to the absence of accurate models and resulting plant-model mismatch, these problems becom…
Reinforcement Learning (RL)