Transfer in Reinforcement Learning via Regret Bounds for Learning Agents

2022-02-02 · Adrienne Tuynman, Ronald Ortner

We present an approach for the quantification of the usefulness of transfer in reinforcement learning via regret bounds for a multi-agent setting. Considering a number of $\aleph$ agents operating in the same Markov decision process, however possibly with different reward functions, we consider the regret each agent suffers with respect to an optimal policy maximizing her average reward. We show that when the agents share their observations the total regret of all agents is smaller by a factor of $\sqrt{\aleph}$ compared to the case when each agent has to rely on the information collected by herself. This result demonstrates how considering the regret in multi-agent settings can provide theoretical bounds on the benefit of sharing observations in transfer learning.

📄 PDF Abstract BibTeX arXiv:2202.01182

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Information-Theoretic Minimax Regret Bounds for Reinforcement Learning based on Duality

2024-10-21 · Raghav Bongole, Amaury Gouverneur, Borja Rodríguez-Gálvez, Tobias J. Oechtering 외

We study agents acting in an unknown environment where the agent's goal is to find a robust policy. We consider robust policies as policies that achieve high cumulative rewards for all possible environments. To this end,…

Sequential Transfer in Multi-armed Bandit with Finite Set of Models

2013-07-25 · NeurIPS 2013 12 · Mohammad Gheshlaghi Azar, Alessandro Lazaric, Emma Brunskill

Learning from prior tasks and transferring that experience to improve future performance is critical for building lifelong learning agents. Although results in supervised and reinforcement learning show that transfer may…

Lifelong learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Theoretically-Grounded Policy Advice from Multiple Teachers in Reinforcement Learning Settings with Applications to Negative Transfer

2016-04-13 · Yusen Zhan, Haitham Bou Ammar, Matthew E. Taylor

Policy advice is a transfer learning method where a student agent is able to learn faster via advice from a teacher. However, both this and other reinforcement learning transfer methods have little theoretical analysis. …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Gap-Dependent Bounds for Nearly Minimax Optimal Reinforcement Learning with Linear Function Approximation

2026-02-23 · Haochen Zhang, Zhong Zheng, Lingzhou Xue arxiv

We study gap-dependent performance guarantees for nearly minimax-optimal algorithms in reinforcement learning with linear function approximation. While prior works have established gap-dependent regret bounds in this set…

Reinforcement Learning

Regret Bounds and Reinforcement Learning Exploration of EXP-based Algorithms

2020-09-20 · Mengfan Xu, Diego Klabjan

We study the challenging exploration incentive problem in both bandit and reinforcement learning, where the rewards are scale-free and potentially unbounded, driven by real-world scenarios and differing from existing wor…

Multi-Armed Bandits reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)