Bridging the Reality Gap of Reinforcement Learning based Traffic Signal Control using Domain Randomization and Meta Learning

2023-07-21 · Arthur Müller, Matthia Sabatelli

Reinforcement Learning (RL) has been widely explored in Traffic Signal Control (TSC) applications, however, still no such system has been deployed in practice. A key barrier to progress in this area is the reality gap, the discrepancy that results from differences between simulation models and their real-world equivalents. In this paper, we address this challenge by first presenting a comprehensive analysis of potential simulation parameters that contribute to this reality gap. We then also examine two promising strategies that can bridge this gap: Domain Randomization (DR) and Model-Agnostic Meta-Learning (MAML). Both strategies were trained with a traffic simulation model of an intersection. In addition, the model was embedded in LemgoRL, a framework that integrates realistic, safety-critical requirements into the control system. Subsequently, we evaluated the performance of the two methods on a separate model of the same intersection that was developed with a different traffic simulator. In this way, we mimic the reality gap. Our experimental results show that both DR and MAML outperform a state-of-the-art RL algorithm, therefore highlighting their potential to mitigate the reality gap in RLbased TSC systems.

📄 PDF Abstract BibTeX arXiv:2307.11357

Code (0)

등록된 구현이 없습니다.

Tasks

Meta-Learning Reinforcement Learning (RL)Traffic Signal Control

Methods 이 논문이 사용한 방법론

MAML 설명 없음

SynTraC: A Synthetic Dataset for Traffic Signal Control from Traffic Monitoring Cameras

2024-08-18 · Tiejin Chen, Prithvi Shirke, Bharatesh Chakravarthi, Arpitsinh Vaghela 외

This paper introduces SynTraC, the first public image-based traffic signal control dataset, aimed at bridging the gap between simulated environments and real-world traffic management challenges. Unlike traditional datase…

Management Traffic Signal Control

The Real Deal: A Review of Challenges and Opportunities in Moving Reinforcement Learning-Based Traffic Signal Control Systems Towards Reality

2022-06-23 · Rex Chen, Fei Fang, Norman Sadeh

Traffic signal control (TSC) is a high-stakes domain that is growing in importance as traffic volume grows globally. An increasing number of works are applying reinforcement learning (RL) to TSC; RL can draw on an abunda…

Reinforcement Learning (RL)Traffic Signal Control

ModelLight: Model-Based Meta-Reinforcement Learning for Traffic Signal Control

2021-11-15 · Xingshuai Huang, Di wu, Michael Jenkin, Benoit Boulet

Traffic signal control is of critical importance for the effective use of transportation infrastructures. The rapid increase of vehicle traffic and changes in traffic patterns make traffic signal control more and more ch…

Meta-Learning Meta Reinforcement Learning reinforcement-learning Reinforcement Learning+2

Bridging Imagination and Reality for Model-Based Deep Reinforcement Learning

2020-10-23 · NeurIPS 2020 12 · Guangxiang Zhu, Minghao Zhang, Honglak Lee, Chongjie Zhang

Sample efficiency has been one of the major challenges for deep reinforcement learning. Recently, model-based reinforcement learning has been proposed to address this challenge by performing planning on imaginary traject…

Deep Reinforcement Learning Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Random Ensemble Reinforcement Learning for Traffic Signal Control

2022-03-10 · Ruijie Qi, Jianbin Huang, He Li, Qinglin Tan 외

Traffic signal control is a significant part of the construction of intelligent transportation. An efficient traffic signal control strategy can reduce traffic congestion, improve urban road traffic efficiency and facili…

Ensemble Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1