Eco-evolutionary Dynamics of Non-episodic Neuroevolution in Large Multi-agent Environments
Neuroevolution (NE) has recently proven a competitive alternative to learning by gradient descent in reinforcement learning tasks. However, the majority of NE methods and associated simulation environments differ crucially from biological evolution: the environment is reset to initial conditions at the end of each generation, whereas natural environments are continuously modified by their inhabitants; agents reproduce based on their ability to maximize rewards within a population, while biological organisms reproduce and die based on internal physiological variables that depend on their resource consumption; simulation environments are primarily single-agent while the biological world is inherently multi-agent and evolves alongside the population. In this work we present a method for continuously evolving adaptive agents without any environment or population reset. The environment is a large grid world with complex spatiotemporal resource generation, containing many agents that are each controlled by an evolvable recurrent neural network and locally reproduce based on their internal physiology. The entire system is implemented in JAX, allowing very fast simulation on a GPU. We show that NE can operate in an ecologically-valid non-episodic multi-agent setting, finding sustainable collective foraging strategies in the presence of a complex interplay between ecological and evolutionary dynamics.
Code (1)
Tasks
GPUvalidSimilar Papers 제목 키워드 기반
VINE: An Open Source Interactive Data Visualization Tool for Neuroevolution
Recent advances in deep neuroevolution have demonstrated that evolutionary algorithms, such as evolution strategies (ES) and genetic algorithms (GA), can scale to train deep neural networks to solve difficult reinforceme…
Data VisualizationEvolutionary AlgorithmsReinforcement LearningReinforcement Learning (RL)Neuroevolutionary Multi-objective approaches to Trajectory Prediction in Autonomous Vehicles
The incentive for using Evolutionary Algorithms (EAs) for the automated optimization and training of deep neural networks (DNNs), a process referred to as neuroevolution, has gained momentum in recent years. The configur…
Autonomous VehiclesEvolutionary AlgorithmsTrajectory PredictionEmergent kin selection of altruistic feeding via non-episodic neuroevolution
Kin selection theory has proven to be a popular and widely accepted account of how altruistic behaviour can evolve under natural selection. Hamilton's rule, first published in 1964, has since been experimentally validate…
A Study of Fitness Landscapes for Neuroevolution
Fitness landscapes are a useful concept to study the dynamics of meta-heuristics. In the last two decades, they have been applied with success to estimate the optimization power of several types of evolutionary algorithm…
Evolutionary AlgorithmsOn the Exploitation of Neuroevolutionary Information: Analyzing the Past for a More Efficient Future
Neuroevolutionary algorithms, automatic searches of neural network structures by means of evolutionary techniques, are computationally costly procedures. In spite of this, due to the great performance provided by the arc…