Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs

2022-07-05 · Benjamin Fuhrer, Yuval Shpigelman, Chen Tessler, Shie Mannor, Gal Chechik, Eitan Zahavi, Gal Dalal

As communication protocols evolve, datacenter network utilization increases. As a result, congestion is more frequent, causing higher latency and packet loss. Combined with the increasing complexity of workloads, manual design of congestion control (CC) algorithms becomes extremely difficult. This calls for the development of AI approaches to replace the human effort. Unfortunately, it is currently not possible to deploy AI models on network devices due to their limited computational capabilities. Here, we offer a solution to this problem by building a computationally-light solution based on a recent reinforcement learning CC algorithm [arXiv:2207.02295]. We reduce the inference time of RL-CC by x500 by distilling its complex neural network into decision trees. This transformation enables real-time inference within the $\mu$-sec decision-time requirement, with a negligible effect on quality. We deploy the transformed policy on NVIDIA NICs in a live cluster. Compared to popular CC algorithms used in production, RL-CC is the only method that performs well on all benchmarks tested over a large range of number of flows. It balances multiple metrics simultaneously: bandwidth, latency, and packet drops. These results suggest that data-driven methods for CC are feasible, challenging the prior belief that handcrafted heuristics are necessary to achieve optimal performance.

📄 PDF Abstract BibTeX arXiv:2207.02295

Code (1)

nvlabs/rlcc 공식 구현 pytorch

Tasks

Fairness reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Reinforcement Learning for Datacenter Congestion Control

2021-02-18 · Chen Tessler, Yuval Shpigelman, Gal Dalal, Amit Mandelbaum 외

We approach the task of network congestion control in datacenters using Reinforcement Learning (RL). Successful congestion control algorithms can dramatically improve latency and overall network throughput. Until today, …

Network Congestion Control reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Impact of RoCE Congestion Control Policies on Distributed Training of DNNs

2022-07-22 · Tarannum Khan, Saeed Rashidi, Srinivas Sridharan, Pallavi Shurpali 외

RDMA over Converged Ethernet (RoCE) has gained significant attraction for datacenter networks due to its compatibility with conventional Ethernet-based fabric. However, the RDMA protocol is efficient only on (nearly) los…

Blocking

Iroko: A Framework to Prototype Reinforcement Learning for Data Center Traffic Control

2018-12-24 · Fabian Ruffy, Michael Przystupa, Ivan Beschastnikh

Recent networking research has identified that data-driven congestion control (CC) can be more efficient than traditional CC in TCP. Deep reinforcement learning (RL), in particular, has the potential to learn optimal net…

Deep Reinforcement Learning OpenAI Gym reinforcement-learning Reinforcement Learning+1

How Much Progress Has There Been in NVIDIA Datacenter GPUs?

2026-01-27 · Emanuele Del Sozzo, Martin Fleming, Kenneth Flamm, Neil Thompson arxiv

As the role of modern Graphics Processing Units (GPUs) becomes increasingly essential for several computing tasks, analyzing their past and current progress is paramount for determining future constraints on scientific r…

Intelligent Coordination among Multiple Traffic Intersections Using Multi-Agent Reinforcement Learning

2019-12-09 · Ujwal Padam Tewari, Vishal Bidawatka, Varsha Raveendran, Vinay Sudhakaran 외

We use Asynchronous Advantage Actor Critic (A3C) for implementing an AI agent in the controllers that optimize flow of traffic across a single intersection and then extend it to multiple intersections by considering a mu…

AI Agent Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1