Assessment of Reward Functions for Reinforcement Learning Traffic Signal Control under Real-World Limitations
Adaptive traffic signal control is one key avenue for mitigating the growing consequences of traffic congestion. Incumbent solutions such as SCOOT and SCATS require regular and time-consuming calibration, can't optimise well for multiple road use modalities, and require the manual curation of many implementation plans. A recent alternative to these approaches are deep reinforcement learning algorithms, in which an agent learns how to take the most appropriate action for a given state of the system. This is guided by neural networks approximating a reward function that provides feedback to the agent regarding the performance of the actions taken, making it sensitive to the specific reward function chosen. Several authors have surveyed the reward functions used in the literature, but attributing outcome differences to reward function choice across works is problematic as there are many uncontrolled differences, as well as different outcome metrics. This paper compares the performance of agents using different reward functions in a simulation of a junction in Greater Manchester, UK, across various demand profiles, subject to real world constraints: realistic sensor inputs, controllers, calibrated demand, intergreen times and stage sequencing. The reward metrics considered are based on the time spent stopped, lost time, change in lost time, average speed, queue length, junction throughput and variations of these magnitudes. The performance of these reward functions is compared in terms of total waiting time. We find that speed maximisation resulted in the lowest average waiting times across all demand levels, displaying significantly better performance than other rewards previously introduced in the literature.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningTraffic Signal ControlSimilar Papers 제목 키워드 기반
Optimizing Traffic Lights with Multi-agent Deep Reinforcement Learning and V2X communication
We consider a system to optimize duration of traffic signals using multi-agent deep reinforcement learning and Vehicle-to-Everything (V2X) communication. This system aims at analyzing independent and shared rewards for m…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Assessment of Reward Functions in Reinforcement Learning for Multi-Modal Urban Traffic Control under Real-World limitations
Reinforcement Learning is proving a successful tool that can manage urban intersections with a fraction of the effort required to curate traditional traffic controllers. However, literature on the introduction and contro…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Large-scale Regional Traffic Signal Control Based on Single-Agent Reinforcement Learning
In the context of global urbanization and motorization, traffic congestion has become a significant issue, severely affecting the quality of life, environment, and economy. This paper puts forward a single-agent reinforc…
Reinforcement Learning (RL)Traffic Signal ControlGenerative Adversarial Reward Learning for Generalized Behavior Tendency Inference
Recent advances in reinforcement learning have inspired increasing interest in learning user modeling adaptively through dynamic interactions, e.g., in reinforcement learning based recommender systems. Reward function is…
Recommendation Systemsreinforcement-learningReinforcement LearningReinforcement Learning (RL)+2Diagnosing Reinforcement Learning for Traffic Signal Control
With the increasing availability of traffic data and advance of deep reinforcement learning techniques, there is an emerging trend of employing reinforcement learning (RL) for traffic signal control. A key question for a…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1