Distributed Proximal Policy Optimization for Contention-Based Spectrum Access

2021-10-07 · Akash Doshi, Jeffrey G. Andrews

The increasing number of wireless devices operating in unlicensed spectrum motivates the development of intelligent adaptive approaches to spectrum access that go beyond traditional carrier sensing. We develop a novel distributed implementation of a policy gradient method known as Proximal Policy Optimization modelled on a two stage Markov decision process that enables such an intelligent approach, and still achieves decentralized contention-based medium access. In each time slot, a base station (BS) uses information from spectrum sensing and reception quality to autonomously decide whether or not to transmit on a given resource, with the goal of maximizing proportional fairness network-wide. Empirically, we find the proportional fairness reward accumulated by the policy gradient approach to be significantly higher than even a genie-aided adaptive energy detection threshold. This is further validated by the improved sum and maximum user throughputs achieved by our approach.

📄 PDF Abstract BibTeX arXiv:2111.09420

Code (0)

등록된 구현이 없습니다.

Tasks

Fairness

Combining Contention-Based Spectrum Access and Adaptive Modulation using Deep Reinforcement Learning

2021-09-24 · Akash Doshi, Jeffrey G. Andrews

The use of unlicensed spectrum for cellular systems to mitigate spectrum scarcity has led to the development of intelligent adaptive approaches to spectrum access that improve upon traditional carrier sensing and listen-…

Deep Reinforcement Learning Fairness reinforcement-learning Reinforcement Learning (RL)

A Deep Reinforcement Learning Framework for Contention-Based Spectrum Sharing

2021-10-05 · Akash Doshi, Srinivas Yerramalli, Lorenzo Ferrari, Taesang Yoo 외

The increasing number of wireless devices operating in unlicensed spectrum motivates the development of intelligent adaptive approaches to spectrum access. We consider decentralized contention-based medium access for bas…

Deep Reinforcement Learning Fairness Q-Learning reinforcement-learning+2

Proximal Policy Optimization with Mixed Distributed Training

2019-07-15 · Zhen-Yu Zhang, Xiangfeng Luo, Tong Liu, Shaorong Xie 외

Instability and slowness are two main problems in deep reinforcement learning. Even if proximal policy optimization (PPO) is the state of the art, it still suffers from these two problems. We introduce an improved algori…

Deep Reinforcement Learning Reinforcement Learning

A Differentiable Digital Twin of Distributed Link Scheduling for Contention-Aware Networking

2025-12-11 · Zhongyuan Zhao, Yujun Ming, Kevin Chan, Ananthram Swami 외 arxiv

Many routing and flow optimization problems in wired networks can be solved efficiently using minimum cost flow formulations. However, this approach does not extend to wireless multi-hop networks, where the assumptions o…

Network Contention-Aware Cluster Scheduling with Reinforcement Learning

2023-10-31 · Junyeol Ryu, Jeongyoon Eo

With continuous advances in deep learning, distributed training is becoming common in GPU clusters. Specifically, for emerging workloads with diverse amounts, ratios, and patterns of communication, we observe that networ…

GPU reinforcement-learning Reinforcement Learning Scheduling