Continual Reinforcement Learning with Complex Synapses

2018-02-20 · ICML 2018 7 · Christos Kaplanis, Murray Shanahan, Claudia Clopath

Unlike humans, who are capable of continual learning over their lifetimes, artificial neural networks have long been known to suffer from a phenomenon known as catastrophic forgetting, whereby new learning can lead to abrupt erasure of previously acquired knowledge. Whereas in a neural network the parameters are typically modelled as scalar values, an individual synapse in the brain comprises a complex network of interacting biochemical components that evolve at different timescales. In this paper, we show that by equipping tabular and deep reinforcement learning agents with a synaptic model that incorporates this biological complexity (Benna & Fusi, 2016), catastrophic forgetting can be mitigated at multiple timescales. In particular, we find that as well as enabling continual learning across sequential training of two simple tasks, it can also be used to overcome within-task forgetting by reducing the need for an experience replay database.

📄 PDF Abstract BibTeX arXiv:1802.07239

Code (0)

등록된 구현이 없습니다.

Tasks

Continual LearningDeep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

Meta-Reinforcement Learning with Self-Modifying Networks

2022-02-04 · Mathieu Chalvidal, Thomas Serre, Rufin VanRullen

Deep Reinforcement Learning has demonstrated the potential of neural networks tuned with gradient descent for solving complex tasks in well-delimited environments. However, these neural systems are slow learners producin…

Deep Reinforcement Learning Meta Reinforcement Learning One-Shot Learning reinforcement-learning+2

Continual Learning Through Synaptic Intelligence

2017-03-13 · ICML 2017 8 · Friedemann Zenke, Ben Poole, Surya Ganguli

While deep learning has led to remarkable advances across diverse applications, it struggles in domains where the data distribution changes over the course of learning. In stark contrast, biological neural networks conti…

Computational Efficiency Continual Learning General Classification

Rewiring Neurons in Non-Stationary Environments

2023-09-21 · NeurIPS 2023 12

The human brain rewires itself for neuroplasticity in the presence of new tasks. We are inspired to harness this key process in continual reinforcement learning, prioritizing adaptation to non-stationary environments. In…

Continual Learning reinforcement-learning Reinforcement Learning

Theories of synaptic memory consolidation and intelligent plasticity for continual learning

2024-05-27 · Friedemann Zenke, Axel Laborieux

Humans and animals learn throughout life. Such continual learning is crucial for intelligence. In this chapter, we examine the pivotal role plasticity mechanisms with complex internal synaptic dynamics could play in enab…

Continual Learning

Continual Learning with Deep Artificial Neurons

2020-11-13 · Blake Camp, Jaya Krishna Mandivarapu, Rolando Estrada

Neurons in real brains are enormously complex computational units. Among other things, they're responsible for transforming inbound electro-chemical vectors into outbound action potentials, updating the strengths of inte…