Evolutionary Action Selection for Gradient-based Policy Learning
Evolutionary Algorithms (EAs) and Deep Reinforcement Learning (DRL) have recently been integrated to take the advantage of the both methods for better exploration and exploitation.The evolutionary part in these hybrid methods maintains a population of policy networks.However, existing methods focus on optimizing the parameters of policy network, which is usually high-dimensional and tricky for EA.In this paper, we shift the target of evolution from high-dimensional parameter space to low-dimensional action space.We propose Evolutionary Action Selection-Twin Delayed Deep Deterministic Policy Gradient (EAS-TD3), a novel hybrid method of EA and DRL.In EAS, we focus on optimizing the action chosen by the policy network and attempt to obtain high-quality actions to promote policy learning through an evolutionary algorithm. We conduct several experiments on challenging continuous control tasks.The result shows that EAS-TD3 shows superior performance over other state-of-art methods.
Code (0)
등록된 구현이 없습니다.
Tasks
continuous-controlContinuous ControlDeep Reinforcement LearningEvolutionary AlgorithmsSimilar Papers 제목 키워드 기반
Guided Sampling-based Evolutionary Deep Neural Network for Intelligent Fault Diagnosis
The diagnostic performance of most of the deep learning models is greatly affected by the selection of model architecture and hyperparameters. Manual selection of model architecture is not feasible as training and evalua…
DiagnosticFault DiagnosisModel SelectionZeroth-Order Actor-Critic: An Evolutionary Framework for Sequential Decision Problems
Evolutionary algorithms (EAs) have shown promise in solving sequential decision problems (SDPs) by simplifying them to static optimization problems and searching for the optimal policy parameters in a zeroth-order way. W…
continuous-controlContinuous ControlEvolutionary AlgorithmsReinforcement Learning (RL)Imitate Optimal Policy: Prevail and Induce Action Collapse in Policy Gradient
Policy gradient (PG) methods in reinforcement learning frequently utilize deep neural networks (DNNs) to learn a shared backbone of feature representations used to compute likelihoods in an action selection layer. Numero…
Reinforcement LearningOpenAI GymNovelty Search for Deep Reinforcement Learning Policy Network Weights by Action Sequence Edit Metric Distance
Reinforcement learning (RL) problems often feature deceptive local optima, and learning methods that optimize purely for reward signal often fail to learn strategies for overcoming them. Deep neuroevolution and novelty s…
Deep Reinforcement LearningEvolutionary AlgorithmsReinforcement LearningReinforcement Learning (RL)Evolutionary Policy Optimization
A key challenge in reinforcement learning (RL) is managing the exploration-exploitation trade-off without sacrificing sample efficiency. Policy gradient (PG) methods excel in exploitation through fine-grained, gradient-b…
Policy Gradient MethodsReinforcement Learning (RL)