Hierarchical Continual Reinforcement Learning via Large Language Model

2024-01-25 · Chaofan Pan, Xin Yang, Hao Wang, Wei Wei, Tianrui Li

The ability to learn continuously in dynamic environments is a crucial requirement for reinforcement learning (RL) agents applying in the real world. Despite the progress in continual reinforcement learning (CRL), existing methods often suffer from insufficient knowledge transfer, particularly when the tasks are diverse. To address this challenge, we propose a new framework, Hierarchical Continual reinforcement learning via large language model (Hi-Core), designed to facilitate the transfer of high-level knowledge. Hi-Core orchestrates a twolayer structure: high-level policy formulation by a large language model (LLM), which represents agenerates a sequence of goals, and low-level policy learning that closely aligns with goal-oriented RL practices, producing the agent's actions in response to the goals set forth. The framework employs feedback to iteratively adjust and verify highlevel policies, storing them along with low-level policies within a skill library. When encountering a new task, Hi-Core retrieves relevant experience from this library to help to learning. Through experiments on Minigrid, Hi-Core has demonstrated its effectiveness in handling diverse CRL tasks, which outperforms popular baselines.

📄 PDF Abstract BibTeX arXiv:2401.15098

Code (0)

등록된 구현이 없습니다.

Tasks

Language ModelingLanguage ModellingLarge Language Modelmodelreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Library 설명 없음

Focus 설명 없음

BASE 설명 없음

Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learning

2024-12-19 · Anthony Kobanda, Rémy Portelas, Odalric-Ambrym Maillard, Ludovic Denoyer

In dynamic domains such as autonomous robotics and video game simulations, agents must continuously adapt to new tasks while retaining previously acquired skills. This ongoing process, known as Continual Reinforcement Le…

Continual Learning MuJoCo reinforcement-learning Reinforcement Learning

Hierarchically Structured Task-Agnostic Continual Learning

2022-11-14 · Heinke Hihn, Daniel A. Braun

One notable weakness of current machine learning algorithms is the poor ability of models to solve new problems without forgetting previously acquired knowledge. The Continual Learning paradigm has emerged as a protocol …

Continual Learning reinforcement-learning Reinforcement Learning (RL)

Mixture-of-Variational-Experts for Continual Learning

2021-10-25 · Heinke Hihn, Daniel A. Braun

One weakness of machine learning algorithms is the poor ability of models to solve new problems without forgetting previously acquired knowledge. The Continual Learning (CL) paradigm has emerged as a protocol to systemat…

Continual Learning Domain-IL Continual Learning Permuted-MNIST reinforcement-learning+4

Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning

2017-12-20 · ICLR 2018 1 · Tianmin Shu, Caiming Xiong, Richard Socher

Learning policies for complex tasks that require multiple different skills is a major challenge in reinforcement learning (RL). It is also a requirement for its deployment in real-world scenarios. This paper proposes a n…

Minecraft reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A Definition of Continual Reinforcement Learning

2023-07-20 · NeurIPS 2023 11 · David Abel, André Barreto, Benjamin Van Roy, Doina Precup 외

In a standard view of the reinforcement learning problem, an agent's goal is to efficiently identify a policy that maximizes long-term reward. However, this perspective is based on a restricted view of learning as findin…

Continual Learning reinforcement-learning Reinforcement Learning