Transfer Reinforcement Learning for Differing Action Spaces via Q-Network Representations

2022-02-05 · Nathan Beck, Abhiramon Rajasekharan, Hieu Tran

Transfer learning approaches in reinforcement learning aim to assist agents in learning their target domains by leveraging the knowledge learned from other agents that have been trained on similar source domains. For example, recent research focus within this space has been placed on knowledge transfer between tasks that have different transition dynamics and reward functions; however, little focus has been placed on knowledge transfer between tasks that have different action spaces. In this paper, we approach the task of transfer learning between domains that differ in action spaces. We present a reward shaping method based on source embedding similarity that is applicable to domains with both discrete and continuous action spaces. The efficacy of our approach is evaluated on transfer to restricted action spaces in the Acrobot-v1 and Pendulum-v0 domains. A comparison with two baselines shows that our method does not outperform these baselines in these continuous action spaces but does show an improvement in these discrete action spaces. We conclude our analysis with future directions for this work.

📄 PDF Abstract BibTeX arXiv:2202.02442

Code (1)

saodem74/Transfer-Learning-in-Reinforcement-Learning 공식 구현 pytorch

Tasks

Acrobotreinforcement-learning Reinforcement Learning (RL)Transfer LearningTransfer Reinforcement Learning

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Cross Domain Policy Transfer with Effect Cycle-Consistency

2024-03-04 · Ruiqi Zhu, Tianhong Dai, Oya Celiktutan

Training a robotic policy from scratch using deep reinforcement learning methods can be prohibitively expensive due to sample inefficiency. To address this challenge, transferring policies trained in the source domain to…

Deep Reinforcement Learning

Growing Action Spaces

2019-06-28 · ICML 2020 1 · Gregory Farquhar, Laura Gustafson, Zeming Lin, Shimon Whiteson 외

In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress. In this work, we use a curriculum of progressively growing action…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Starcraft

Transferring Expectations in Model-based Reinforcement Learning

2012-12-01 · NeurIPS 2012 12 · Trung Nguyen, Tomi Silander, Tze Y. Leong

We study how to automatically select and adapt multiple abstractions or representations of the world to support model-based reinforcement learning. We address the challenges of transfer learning in heterogeneous environm…

model Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning+2

ASALT: Adaptive State Alignment for Lateral Transfer in Multi-agent Reinforcement Learning

2026-06-23 · Anurag Akula, Satheesh K. Perepu, Abhishek Sarkar, Kaushik Dey arxiv

Multi-agent reinforcement learning (MARL) addresses the problem of training multiple agents that pursue collaborative, competitive, or mixed objectives. Prior work has investigated transfer learning between source and ta…

Multi-agent Reinforcement Learning Transfer Learning

Abstracting Geo-specific Terrains to Scale Up Reinforcement Learning

2025-03-25 · Volkan Ustun, Soham Hans, Rajay Kumar, Yunzhe Wang

Multi-agent reinforcement learning (MARL) is increasingly ubiquitous in training dynamic and adaptive synthetic characters for interactive simulations on geo-specific terrains. Frameworks such as Unity's ML-Agents help t…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning