Evolutionary Reinforcement Learning via Cooperative Coevolutionary Negatively Correlated Search
Evolutionary algorithms (EAs) have been successfully applied to optimize the policies for Reinforcement Learning (RL) tasks due to their exploration ability. The recently proposed Negatively Correlated Search (NCS) provides a distinct parallel exploration search behavior and is expected to facilitate RL more effectively. Considering that the commonly adopted neural policies usually involves millions of parameters to be optimized, the direct application of NCS to RL may face a great challenge of the large-scale search space. To address this issue, this paper presents an NCS-friendly Cooperative Coevolution (CC) framework to scale-up NCS while largely preserving its parallel exploration search behavior. The issue of traditional CC that can deteriorate NCS is also discussed. Empirical studies on 10 popular Atari games show that the proposed method can significantly outperform three state-of-the-art deep RL methods with 50% less computational time by effectively exploring a 1.7 million-dimensional search space.
Code (0)
등록된 구현이 없습니다.
Tasks
Atari GamesEvolutionary Algorithmsreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
New Pathways in Coevolutionary Computation
The simultaneous evolution of two or more species with coupled fitness -- coevolution -- has been put to good use in the field of evolutionary computation. Herein, we present two new forms of coevolutionary algorithms, w…
Codynamic Fitness Landscapes of Coevolutionary Minimal Substrates
Coevolutionary minimal substrates are simple and abstract models that allow studying the relationships and codynamics between objective and subjective fitness. Using these models an approach is presented for defining and…
A Cooperative Coevolutionary Genetic Algorithm for Learning Bayesian Network Structures
We propose a cooperative coevolutionary genetic algorithm for learning Bayesian network structures from fully observable data sets. Since this problem can be decomposed into two dependent subproblems, that is to find an …
Evolutionary Reinforcement Learning via Cooperative Coevolution
Recently, evolutionary reinforcement learning has obtained much attention in various domains. Maintaining a population of actors, evolutionary reinforcement learning utilises the collected experiences to improve the beha…
Efficient Explorationreinforcement-learningReinforcement LearningA coevolutionary approach to deep multi-agent reinforcement learning
Traditionally, Deep Artificial Neural Networks (DNN's) are trained through gradient descent. Recent research shows that Deep Neuroevolution (DNE) is also capable of evolving multi-million-parameter DNN's, which proved to…
Atari GamesDecision MakingMulti-agent Reinforcement Learningreinforcement-learning+2