홈 › Papers

Statistical Guarantees for Lifelong Reinforcement Learning using PAC-Bayes Theory

2024-11-01 · Zhi Zhang, Chris Chow, Yasi Zhang, Yanchao Sun, Haochen Zhang, Eric Hanchen Jiang, Han Liu, Furong Huang, Yuchen Cui, Oscar Hernan Madrid Padilla

Lifelong reinforcement learning (RL) has been developed as a paradigm for extending single-task RL to more realistic, dynamic settings. In lifelong RL, the "life" of an RL agent is modeled as a stream of tasks drawn from a task distribution. We propose EPIC (Empirical PAC-Bayes that Improves Continuously), a novel algorithm designed for lifelong RL using PAC-Bayes theory. EPIC learns a shared policy distribution, referred to as the world policy, which enables rapid adaptation to new tasks while retaining valuable knowledge from previous experiences. Our theoretical analysis establishes a relationship between the algorithm's generalization performance and the number of prior tasks preserved in memory. We also derive the sample complexity of EPIC in terms of RL regret. Extensive experiments on a variety of environments demonstrate that EPIC significantly outperforms existing methods in lifelong RL, offering both theoretical guarantees and practical efficacy through the use of the world policy.

📄 PDF Abstract BibTeX arXiv:2411.00401

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Bayesian Exploration for Lifelong Reinforcement Learning

2021-09-29 · Haotian Fu, Shangqun Yu, Michael Littman, George Konidaris

A central question in reinforcement learning (RL) is how to leverage prior knowledge to accelerate learning in new tasks. We propose a Bayesian exploration method for lifelong reinforcement learning (BLRL) that aims to …

Lifelong learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

State Abstractions for Lifelong Reinforcement Learning

2018-07-01 · ICML 2018 7 · David Abel, Dilip Arumugam, Lucas Lehnert, Michael Littman

In lifelong reinforcement learning, agents must effectively transfer knowledge across tasks while simultaneously addressing exploration, credit assignment, and generalization. State abstraction can help overcome the…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Model-based Lifelong Reinforcement Learning with Bayesian Exploration

2022-10-20 · Haotian Fu, Shangqun Yu, Michael Littman, George Konidaris

We propose a model-based lifelong reinforcement-learning approach that estimates a hierarchical Bayesian posterior distilling the common structure shared across different tasks. The learned posterior combined with a samp…

model reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Statistical Guarantees for Variational Autoencoders using PAC-Bayesian Theory

2023-10-07 · NeurIPS 2023 11 · Sokhna Diarra Mbacke, Florence Clerc, Pascal Germain

Since their inception, Variational Autoencoders (VAEs) have become central in machine learning. Despite their widespread use, numerous questions regarding their theoretical properties remain open. Using PAC-Bayesian theo…

Multi-Agent Path Finding Among Dynamic Uncontrollable Agents with Statistical Safety Guarantees

2025-07-29 · Kegan J. Strawn, Thomy Phan, Eric Wang, Nora Ayanian 외 arxiv

Existing multi-agent path finding (MAPF) solvers do not account for uncertain behavior of uncontrollable agents. We present a novel variant of Enhanced Conflict-Based Search (ECBS), for both one-shot and lifelong MAPF in…