Generating Behaviorally Diverse Policies with Latent Diffusion Models

2023-05-30 · NeurIPS 2023 11

Recent progress in Quality Diversity Reinforcement Learning (QD-RL) has enabled learning a collection of behaviorally diverse, high performing policies. However, these methods typically involve storing thousands of policies, which results in high space-complexity and poor scaling to additional behaviors. Condensing the archive into a single model while retaining the performance and coverage of the original collection of policies has proved challenging. In this work, we propose using diffusion models to distill the archive into a single generative model over policy parameters. We show that our method achieves a compression ratio of 13x while recovering 98% of the original rewards and 89% of the original coverage. Further, the conditioning mechanism of diffusion models allows for flexibly selecting and sequencing behaviors, including using language. Project website: https://sites.google.com/view/policydiffusion/home

📄 PDF Abstract BibTeX arXiv:2305.18738

Code (0)

등록된 구현이 없습니다.

Tasks

Diversity

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learning

2026-04-16 · Lute Lillo, Nick Cheney arxiv

Continual reinforcement learning must balance retention with adaptation, yet many methods still rely on \emph{single-model preservation}, committing to one evolving policy as the main reusable solution across tasks. Even…

Reinforcement Learning

Latent Weight Diffusion: Generating reactive policies instead of trajectories

2024-10-17 · Shashank Hegde, Satyajeet Das, Gautam Salhotra, Gaurav S. Sukhatme

With the increasing availability of open-source robotic data, imitation learning has emerged as a viable approach for both robot manipulation and locomotion. Currently, large generalized policies are trained to predict c…

Denoising Imitation Learning Robot Manipulation

Safety-Critical Traffic Simulation with Guided Latent Diffusion Model

2025-05-01 · Mingxing Peng, Ruoyu Yao, Xusen Guo, Yuting Xie 외

Safety-critical traffic simulation plays a crucial role in evaluating autonomous driving systems under rare and challenging scenarios. However, existing approaches often generate unrealistic scenarios due to insufficient…

Autonomous Driving Computational Efficiency Denoising

Structure-Conditioned Actor-Critic Branches for Quality-Diversity Reinforcement Learning

2026-06-07 · Lianrong Zuo, Peilan Xu, Yong Liu, Wenjian Luo arxiv

Quality-diversity reinforcement learning (QD-RL) aims to construct policy repertoires that contain both high-performing and behaviorally diverse policies. Existing QD-RL methods mainly diversify policy instances after ro…