Efficient Replay Memory Architectures in Multi-Agent Reinforcement Learning for Traffic Congestion Control

2024-07-22 · Mukul Chodhary, Kevin Octavian, SooJean Han

Episodic control, inspired by the role of episodic memory in the human brain, has been shown to improve the sample inefficiency of model-free reinforcement learning by reusing high-return past experiences. However, the memory growth of episodic control is undesirable in large-scale multi-agent problems such as vehicle traffic management. This paper proposes a novel replay memory architecture called Dual-Memory Integrated Learning, to augment to multi-agent reinforcement learning methods for congestion control via adaptive light signal scheduling. Our dual-memory architecture mimics two core capabilities of human decision-making. First, it relies on diverse types of memory--semantic and episodic, short-term and long-term--in order to remember high-return states that occur often in the network and filter out states that don't. Second, it employs equivalence classes to group together similar state-action pairs and that can be controlled using the same action (i.e., light signal sequence). Theoretical analyses establish memory growth bounds, and simulation experiments on several intersection networks showcase improved congestion performance (e.g., vehicle throughput) from our method.

📄 PDF Abstract BibTeX arXiv:2407.16034

Code (0)

등록된 구현이 없습니다.

Tasks

Decision Making Multi-agent Reinforcement Learning reinforcement-learning Reinforcement LearningScheduling

Associative Memory Based Experience Replay for Deep Reinforcement Learning

2022-07-16 · Mengyuan Li, Arman Kazemi, Ann Franchesca Laguna, X. Sharon Hu

Experience replay is an essential component in deep reinforcement learning (DRL), which stores the experiences and generates experiences for the agent to learn in real time. Recently, prioritized experience replay (PER) …

CPU Deep Reinforcement Learning GPU reinforcement-learning+2

Proxy Experience Replay: Federated Distillation for Distributed Reinforcement Learning

2020-05-13 · Han Cha, Jihong Park, Hyesung Kim, Mehdi Bennis 외

Traditional distributed deep reinforcement learning (RL) commonly relies on exchanging the experience replay memory (RM) of each agent. Since the RM contains all state observations and action policy history, it may incur…

Clustering Data Augmentation Deep Reinforcement Learning Privacy Preserving+3

Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning

2017-02-28 · ICML 2017 8 · Jakob Foerster, Nantas Nardelli, Gregory Farquhar, Triantafyllos Afouras 외

Many real-world problems, such as network packet routing and urban traffic control, are naturally modeled as multi-agent reinforcement learning (RL) problems. However, existing multi-agent RL methods typically scale poor…

Multi-agent Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+2

The Effects of Memory Replay in Reinforcement Learning

2017-10-18 · Ruishan Liu, James Zou

Experience replay is a key technique behind many recent advances in deep reinforcement learning. Allowing the agent to learn from earlier memories can speed up learning and break undesirable temporal correlations. Despit…

Deep Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+1

A Dual Memory Structure for Efficient Use of Replay Memory in Deep Reinforcement Learning

2019-07-15 · Wonshick Ko, Dong Eui Chang

In this paper, we propose a dual memory structure for reinforcement learning algorithms with replay memory. The dual memory consists of a main memory that stores various data and a cache memory that manages the data and …

Deep Reinforcement Learning OpenAI Gym reinforcement-learning Reinforcement Learning+1