Sequential Transfer in Reinforcement Learning with a Generative Model

2020-07-01 · ICML 2020 1 · Andrea Tirinzoni, Riccardo Poiani, Marcello Restelli

We are interested in how to design reinforcement learning agents that provably reduce the sample complexity for learning new tasks by transferring knowledge from previously-solved ones. The availability of solutions to related problems poses a fundamental trade-off: whether to seek policies that are expected to achieve high (yet sub-optimal) performance in the new task immediately or whether to seek information to quickly identify an optimal solution, potentially at the cost of poor initial behavior. In this work, we focus on the second objective when the agent has access to a generative model of state-action pairs. First, given a set of solved tasks containing an approximation of the target one, we design an algorithm that quickly identifies an accurate solution by seeking the state-action pairs that are most informative for this purpose. We derive PAC bounds on its sample complexity which clearly demonstrate the benefits of using this kind of prior knowledge. Then, we show how to learn these approximate tasks sequentially by reducing our transfer setting to a hidden Markov model and employing spectral methods to recover its parameters. Finally, we empirically verify our theoretical findings in simple simulated domains.

📄 PDF Abstract BibTeX arXiv:2007.00722

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Decision Stacks: Flexible Reinforcement Learning via Modular Generative Models

2023-06-09 · NeurIPS 2023 11

Reinforcement learning presents an attractive paradigm to reason about several distinct aspects of sequential decision making, such as specifying complex goals, planning future observations and actions, and critiquing th…

Decision Making reinforcement-learning Reinforcement Learning Sequential Decision Making

Continual Offline Reinforcement Learning via Diffusion-based Dual Generative Replay

2024-04-16 · Jinmei Liu, Wenbin Li, Xiangyu Yue, Shilin Zhang 외

We study continual offline reinforcement learning, a practical paradigm that facilitates forward transfer and mitigates catastrophic forgetting to tackle sequential offline tasks. We propose a dual generative replay fram…

Continual Learning reinforcement-learning Reinforcement Learning

Transfer Learning in Deep Reinforcement Learning: A Survey

2020-09-16 · Zhuangdi Zhu, Kaixiang Lin, Anil K. Jain, Jiayu Zhou

Reinforcement learning is a learning paradigm for solving sequential decision-making problems. Recent years have witnessed remarkable progress in reinforcement learning upon the fast development of deep neural networks. …

Decision Making Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+4

Sequential Transfer in Multi-armed Bandit with Finite Set of Models

2013-07-25 · NeurIPS 2013 12 · Mohammad Gheshlaghi Azar, Alessandro Lazaric, Emma Brunskill

Learning from prior tasks and transferring that experience to improve future performance is critical for building lifelong learning agents. Although results in supervised and reinforcement learning show that transfer may…

Lifelong learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Knowledge Transfer in Deep Reinforcement Learning via an RL-Specific GAN-Based Correspondence Function

2022-09-14 · Marko Ruman, Tatiana V. Guy

Deep reinforcement learning has demonstrated superhuman performance in complex decision-making tasks, but it struggles with generalization and knowledge reuse - key aspects of true intelligence. This article introduces a…

Decision Making Deep Reinforcement Learning Generative Adversarial Network reinforcement-learning+3