Continual Reinforcement Learning with Multi-Timescale Replay

2020-04-16 · Christos Kaplanis, Claudia Clopath, Murray Shanahan

In this paper, we propose a multi-timescale replay (MTR) buffer for improving continual learning in RL agents faced with environments that are changing continuously over time at timescales that are unknown to the agent. The basic MTR buffer comprises a cascade of sub-buffers that accumulate experiences at different timescales, enabling the agent to improve the trade-off between adaptation to new data and retention of old knowledge. We also combine the MTR framework with invariant risk minimization, with the idea of encouraging the agent to learn a policy that is robust across the various environments it encounters over time. The MTR methods are evaluated in three different continual learning settings on two continuous control tasks and, in many cases, show improvement over the baselines.

📄 PDF Abstract BibTeX arXiv:2004.07530

Code (1)

ChristosKap/multi_timescale_replay 공식 구현

Tasks

Continual Learning continuous-control Continuous Control reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Continual Reinforcement Learning with Complex Synapses

2018-02-20 · ICML 2018 7 · Christos Kaplanis, Murray Shanahan, Claudia Clopath

Unlike humans, who are capable of continual learning over their lifetimes, artificial neural networks have long been known to suffer from a phenomenon known as catastrophic forgetting, whereby new learning can lead to ab…

Continual Learning Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Local Timescale Gates for Timescale-Robust Continual Spiking Neural Networks

2025-10-13 · Ansh Tiwari, Ayush Chauhan arxiv

Spiking neural networks (SNNs) promise energy-efficient artificial intelligence on neuromorphic hardware but struggle with tasks requiring both fast adaptation and long-term memory, especially in continual learning. We p…

Continual Learning

Replay-enhanced Continual Reinforcement Learning

2023-11-20 · Tiantian Zhang, Kevin Zehua Shen, Zichuan Lin, Bo Yuan 외

Replaying past experiences has proven to be a highly effective approach for averting catastrophic forgetting in supervised continual learning. However, some crucial factors are still largely ignored, making it vulnerable…

Continual Learning reinforcement-learning Reinforcement Learning

Learn the Time to Learn: Replay Scheduling in Continual Learning

2022-09-18 · Marcus Klasson, Hedvig Kjellström, Cheng Zhang

Replay methods are known to be successful at mitigating catastrophic forgetting in continual learning scenarios despite having limited access to historical data. However, storing historical data is cheap in many real-wor…

Continual Learning Scheduling

The Effectiveness of World Models for Continual Reinforcement Learning

2022-11-29 · Samuel Kessler, Mateusz Ostaszewski, Michał Bortkiewicz, Mateusz Żarski 외

World models power some of the most efficient reinforcement learning algorithms. In this work, we showcase that they can be harnessed for continual learning - a situation when the agent faces changing environments. World…

Continual Learning Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning+1