홈 › Papers

Reinforcement Learning for Datacenter Congestion Control

2021-02-18 · Chen Tessler, Yuval Shpigelman, Gal Dalal, Amit Mandelbaum, Doron Haritan Kazakov, Benjamin Fuhrer, Gal Chechik, Shie Mannor

We approach the task of network congestion control in datacenters using Reinforcement Learning (RL). Successful congestion control algorithms can dramatically improve latency and overall network throughput. Until today, no such learning-based algorithms have shown practical potential in this domain. Evidently, the most popular recent deployments rely on rule-based heuristics that are tested on a predetermined set of benchmarks. Consequently, these heuristics do not generalize well to newly-seen scenarios. Contrarily, we devise an RL-based algorithm with the aim of generalizing to different configurations of real-world datacenter networks. We overcome challenges such as partial-observability, non-stationarity, and multi-objectiveness. We further propose a policy gradient algorithm that leverages the analytical structure of the reward function to approximate its derivative and improve stability. We show that this scheme outperforms alternative popular RL approaches, and generalizes to scenarios that were not seen during training. Our experiments, conducted on a realistic simulator that emulates communication networks' behavior, exhibit improved performance concurrently on the multiple considered metrics compared to the popular algorithms deployed today in real datacenters. Our algorithm is being productized to replace heuristics in some of the largest datacenters in the world.

📄 PDF Abstract BibTeX arXiv:2102.09337

Code (0)

등록된 구현이 없습니다.

Tasks

Network Congestion Controlreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Impact of RoCE Congestion Control Policies on Distributed Training of DNNs

2022-07-22 · Tarannum Khan, Saeed Rashidi, Srinivas Sridharan, Pallavi Shurpali 외

RDMA over Converged Ethernet (RoCE) has gained significant attraction for datacenter networks due to its compatibility with conventional Ethernet-based fabric. However, the RDMA protocol is efficient only on (nearly) los…

Blocking

Iroko: A Framework to Prototype Reinforcement Learning for Data Center Traffic Control

2018-12-24 · Fabian Ruffy, Michael Przystupa, Ivan Beschastnikh

Recent networking research has identified that data-driven congestion control (CC) can be more efficient than traditional CC in TCP. Deep reinforcement learning (RL), in particular, has the potential to learn optimal net…

Deep Reinforcement Learning OpenAI Gym reinforcement-learning Reinforcement Learning+1

Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs

2022-07-05 · Benjamin Fuhrer, Yuval Shpigelman, Chen Tessler, Shie Mannor 외

As communication protocols evolve, datacenter network utilization increases. As a result, congestion is more frequent, causing higher latency and packet loss. Combined with the increasing complexity of workloads, manual …

Fairness reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

GraphCC: A Practical Graph Learning-based Approach to Congestion Control in Datacenters

2023-08-09 · Guillermo Bernárdez, José Suárez-Varela, Xiang Shi, Shihan Xiao 외

Congestion Control (CC) plays a fundamental role in optimizing traffic in Data Center Networks (DCN). Currently, DCNs mainly implement two main CC protocols: DCTCP and DCQCN. Both protocols -- and their main variants -- …

Graph Learning Multi-agent Reinforcement Learning

Job Scheduling in Datacenters using Constraint Controlled RL

2022-11-10 · Vanamala Venkataswamy

This paper studies a model for online job scheduling in green datacenters. In green datacenters, resource availability depends on the power supply from the renewables. Intermittent power supply from renewables leads to i…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)Scheduling