Mitigating Partial Observability in Adaptive Traffic Signal Control with Transformers

2024-09-16 · Xiaoyu Wang, Ayal Taitler, Scott Sanner, Baher Abdulhai

Efficient traffic signal control is essential for managing urban transportation, minimizing congestion, and improving safety and sustainability. Reinforcement Learning (RL) has emerged as a promising approach to enhancing adaptive traffic signal control (ATSC) systems, allowing controllers to learn optimal policies through interaction with the environment. However, challenges arise due to partial observability (PO) in traffic networks, where agents have limited visibility, hindering effectiveness. This paper presents the integration of Transformer-based controllers into ATSC systems to address PO effectively. We propose strategies to enhance training efficiency and effectiveness, demonstrating improved coordination capabilities in real-world scenarios. The results showcase the Transformer-based model's ability to capture significant information from historical observations, leading to better control policies and improved traffic flow. This study highlights the potential of leveraging the advanced Transformer architecture to enhance urban transportation management.

📄 PDF Abstract BibTeX arXiv:2409.10693

Code (0)

등록된 구현이 없습니다.

Tasks

Management Reinforcement Learning (RL)Traffic Signal Control

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Multi-Head Attention 설명 없음

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Attention 설명 없음

Quantifying the Impact of Non-Stationarity in Reinforcement Learning-Based Traffic Signal Control

2020-04-09 · Lucas N. Alegre, Ana L. C. Bazzan, Bruno C. da Silva

In reinforcement learning (RL), dealing with non-stationarity is a challenging issue. However, some domains such as traffic optimization are inherently non-stationary. Causes for and effects of this are manifold. In part…

Reinforcement Learning Reinforcement Learning (RL)Traffic Signal Control

Benchmarking Partial Observability in Reinforcement Learning with a Suite of Memory-Improvable Domains

2025-07-31 · Ruo Yu Tao, Kaicheng Guo, Cameron Allen, George Konidaris arxiv

Mitigating partial observability is a necessary but challenging task for general reinforcement learning algorithms. To improve an algorithm's ability to mitigate partial observability, researchers need comprehensive benc…

General Reinforcement Learning

UCATSC: Uncertainty-Aware Constrained Traffic Signal Control Under Vision-Based Partial Observability

2026-02-08 · Jayawant Bodagala, Balaji Bodagala arxiv

Camera-based adaptive traffic signal control is inherently partially observable: detections can be missed, vehicle speeds and distances can be noisy, and a phase-change decision becomes temporally irreversible once yello…

Partially Detected Intelligent Traffic Signal Control: Environmental Adaptation

2019-10-23 · Rusheng Zhang, Romain Leteurtre, Benjamin Striner, Ammar Alanazi 외

Partially Detected Intelligent Traffic Signal Control (PD-ITSC) systems that can optimize traffic signals based on limited detected information could be a cost-efficient solution for mitigating traffic congestion in the …

Q-Learning Reinforcement Learning Reinforcement Learning (RL)Traffic Signal Control+1

Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control

2019-03-11 · Tianshu Chu, Jie Wang, Lara Codecà, Zhaojian Li

Reinforcement learning (RL) is a promising data-driven approach for adaptive traffic signal control (ATSC) in complex urban traffic networks, and deep neural networks further enhance its learning power. However, centrali…

Deep Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+2