A Deep Multi-Agent Reinforcement Learning Approach to Autonomous Separation Assurance

2020-03-17 · Marc Brittain, Xuxi Yang, Peng Wei

A novel deep multi-agent reinforcement learning framework is proposed to identify and resolve conflicts among a variable number of aircraft in a high-density, stochastic, and dynamic sector. Currently the sector capacity is constrained by human air traffic controller's cognitive limitation. We investigate the feasibility of a new concept (autonomous separation assurance) and a new approach to push the sector capacity above human cognitive limitation. We propose the concept of using distributed vehicle autonomy to ensure separation, instead of a centralized sector air traffic controller. Our proposed framework utilizes Proximal Policy Optimization (PPO) that we modify to incorporate an attention network. This allows the agents to have access to variable aircraft information in the sector in a scalable, efficient approach to achieve high traffic throughput under uncertainty. Agents are trained using a centralized learning, decentralized execution scheme where one neural network is learned and shared by all agents. The proposed framework is validated on three challenging case studies in the BlueSky air traffic control environment. Numerical results show the proposed framework significantly reduces offline training time, increases performance, and results in a more efficient policy.

📄 PDF Abstract BibTeX arXiv:2003.08353

Code (2)

suryakmurthy/uam_noise_aware_agent

suryakmurthy/uam_shared_scheduling

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Safety Enhancement for Deep Reinforcement Learning in Autonomous Separation Assurance

2021-05-05 · Wei Guo, Marc Brittain, Peng Wei

The separation assurance task will be extremely challenging for air traffic controllers in a complex and high density airspace environment. Deep reinforcement learning (DRL) was used to develop an autonomous separation a…

Data Augmentation Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Improving Autonomous Separation Assurance through Distributed Reinforcement Learning with Attention Networks

2023-08-09 · Marc W. Brittain, Luis E. Alvarez, Kara Breeden

Advanced Air Mobility (AAM) introduces a new, efficient mode of transportation with the use of vehicle autonomy and electrified aircraft to provide increasingly autonomous transportation between previously underserved ma…

Distributed Computing reinforcement-learning

Transformer-based Multi-agent Reinforcement Learning for Separation Assurance in Structured and Unstructured Airspaces

2026-01-07 · Arsyi Aziz, Peng Wei arxiv

Conventional optimization-based metering depends on strict adherence to precomputed schedules, which limits the flexibility required for the stochastic operations of Advanced Air Mobility (AAM). In contrast, multi-agent …

Multi-agent Reinforcement Learning

A Reinforcement Learning Approach to Quiet and Safe UAM Traffic Management

2025-01-15 · Surya Murthy, John-Paul Clarke, Ufuk Topcu, Zhenyu Gao

Urban air mobility (UAM) is a transformative system that operates various small aerial vehicles in urban environments to reshape urban transportation. However, integrating UAM into existing urban environments presents a …

Management Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Robust Multi-Agent Reinforcement Learning for Small UAS Separation Assurance under GPS Degradation and Spoofing

2026-03-30 · Alex Zongo, Filippos Fotiadis, Ufuk Topcu, Peng Wei arxiv

We address robust separation assurance for small Unmanned Aircraft Systems (sUAS) under GPS degradation and spoofing via Multi-Agent Reinforcement Learning (MARL). In cooperative surveillance, each aircraft (or agent) br…