Sample efficient Quality Diversity for neural continuous control
We propose a novel Deep Neuroevolution algorithm, QD-RL, that combines the strengths of off-policy reinforcement learning (RL) algorithms and Quality Diversity (QD) approaches to solve continuous control problems with neural controllers. The QD part contributes structural biases by decoupling the search for diversity from the search for high return, resulting in efficient management of the exploration-exploitation trade-off. The RL part contributes sample efficiency by relying on off-policy gradient-based updates of the agents. More precisely, we train a population of off-policy deep RL agents to simultaneously maximize diversity within the population and the return of each individual agent. QD-RL selects agents interchangeably from a Pareto front or from a Map-Elites grid, resulting in stable and efficient population updates. Our experiments on the AntMaze and AntTrap environments show that QD-RL can solve challenging exploration and control problems with deceptive rewards while being over 15 times more sample efficient than its evolutionary counterparts.
Code (0)
등록된 구현이 없습니다.
Tasks
continuous-controlContinuous ControlDiversityManagementReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Diversity Policy Gradient for Sample Efficient Quality-Diversity Optimization
A fascinating aspect of nature lies in its ability to produce a large and diverse collection of organisms that are all high-performing in their niche. By contrast, most AI algorithms focus on finding a single efficient s…
continuous-controlContinuous ControlDiversityEvolutionary AlgorithmsPoint Cloud Generation with Continuous Conditioning
Generative models can be used to synthesize 3D objects of high quality and diversity. However, there is typically no control over the properties of the generated object.This paper proposes a novel generative adversarial …
Density EstimationDiversityGenerative Adversarial NetworkPoint Cloud GenerationImproving Generative Adversarial Networks via Adversarial Learning in Latent Space
Generative Adversarial Networks (GANs) have been widely studied as generative models, which map a latent distribution to the target distribution. Although many efforts have been made in terms of backbone architecture des…
DiversityUnlocking the Capabilities of Masked Generative Models for Image Synthesis via Self-Guidance
Masked generative models (MGMs) have shown impressive generative ability while providing an order of magnitude efficient sampling steps compared to continuous diffusion models. However, MGMs still underperform in image s…
DiversityImage Generationparameter-efficient fine-tuningUnderstanding the Quality-Diversity Trade-off in Diffusion Language Models
Diffusion models have seen immense success in modelling continuous data across a range of domains such as vision and audio. Despite the challenges of adapting diffusion models to discrete data, recent work explores their…
DiversityLanguage ModelingLanguage ModellingText Generation