SocialLight: Distributed Cooperation Learning towards Network-Wide Traffic Signal Control
Many recent works have turned to multi-agent reinforcement learning (MARL) for adaptive traffic signal control to optimize the travel time of vehicles over large urban networks. However, achieving effective and scalable cooperation among junctions (agents) remains an open challenge, as existing methods often rely on extensive, non-generalizable reward shaping or on non-scalable centralized learning. To address these problems, we propose a new MARL method for traffic signal control, SocialLight, which learns cooperative traffic control policies by distributedly estimating the individual marginal contribution of agents on their local neighborhood. SocialLight relies on the Asynchronous Actor Critic (A3C) framework, and makes learning scalable by learning a locally-centralized critic conditioned over the states and actions of neighboring agents, used by agents to estimate individual contributions by counterfactual reasoning. We further introduce important modifications to the advantage calculation that help stabilize policy updates. These modifications decouple the impact of the neighbors' actions on the computed advantages, thereby reducing the variance in the gradient updates. We benchmark our trained network against state-of-the-art traffic signal control methods on standard benchmarks in two traffic simulators, SUMO and CityFlow. Our results show that SocialLight exhibits improved scalability to larger road networks and better performance across usual traffic metrics.
Code (0)
등록된 구현이 없습니다.
Tasks
counterfactualCounterfactual ReasoningMulti-agent Reinforcement LearningTraffic Signal ControlMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
CoLight: Learning Network-level Cooperation for Traffic Signal Control
Cooperation among the traffic signals enables vehicles to move through intersections more quickly. Conventional transportation approaches implement cooperation by pre-calculating the offsets between two intersections. Su…
Multi-agent Reinforcement LearningReinforcement LearningTraffic Signal ControlMulti-lane Unsignalized Intersection Cooperation with Flexible Lane Direction based on Multi-vehicle Formation Control
Unsignalized intersection cooperation of connected and automated vehicles (CAVs) is able to eliminate green time loss of signalized intersections and improve traffic efficiency. Most of the existing research on unsignali…
Feudal Multi-Agent Reinforcement Learning with Adaptive Network Partition for Traffic Signal Control
Multi-agent reinforcement learning (MARL) has been applied and shown great potential in multi-intersections traffic signal control, where multiple agents, one for each intersection, must cooperate together to optimize tr…
Graph Neural NetworkMulti-agent Reinforcement Learningreinforcement-learningReinforcement Learning (RL)+1A Novel Multi-Agent Deep RL Approach for Traffic Signal Control
As travel demand increases and urban traffic condition becomes more complicated, applying multi-agent deep reinforcement learning (MARL) to traffic signal control becomes one of the hot topics. The rise of Reinforcement …
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1Perimeter Control with Heterogeneous Metering Rates for Cordon Signals: A Physics-Regularized Multi-Agent Reinforcement Learning Approach
Perimeter Control (PC) strategies have been proposed to address urban road network control in oversaturated situations by regulating the transfer flow of the Protected Network (PN) based on the Macroscopic Fundamental Di…
Multi-agent Reinforcement LearningTraffic Signal Control