A Scalable Deep Reinforcement Learning Model for Online Scheduling Coflows of Multi-Stage Jobs for High Performance Computing

2021-12-21 · Xin Wang, Hong Shen

Coflow is a recently proposed networking abstraction to help improve the communication performance of data-parallel computing jobs. In multi-stage jobs, each job consists of multiple coflows and is represented by a Directed Acyclic Graph (DAG). Efficiently scheduling coflows is critical to improve the data-parallel computing performance in data centers. Compared with hand-tuned scheduling heuristics, existing work DeepWeave [1] utilizes Reinforcement Learning (RL) framework to generate highly-efficient coflow scheduling policies automatically. It employs a graph neural network (GNN) to encode the job information in a set of embedding vectors, and feeds a flat embedding vector containing the whole job information to the policy network. However, this method has poor scalability as it is unable to cope with jobs represented by DAGs of arbitrary sizes and shapes, which requires a large policy network for processing a high-dimensional embedding vector that is difficult to train. In this paper, we first utilize a directed acyclic graph neural network (DAGNN) to process the input and propose a novel Pipelined-DAGNN, which can effectively speed up the feature extraction process of the DAGNN. Next, we feed the embedding sequence composed of schedulable coflows instead of a flat embedding of all coflows to the policy network, and output a priority sequence, which makes the size of the policy network depend on only the dimension of features instead of the product of dimension and number of nodes in the job's DAG.Furthermore, to improve the accuracy of the priority scheduling policy, we incorporate the Self-Attention Mechanism into a deep RL model to capture the interaction between different parts of the embedding sequence to make the output priority scores relevant. Based on this model, we then develop a coflow scheduling algorithm for online multi-stage jobs.

📄 PDF Abstract BibTeX arXiv:2112.11055

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningGraph Neural Network Reinforcement Learning (RL)Scheduling

Methods 이 논문이 사용한 방법론

Graph Neural Network 설명 없음

DAGNN A GNN for dags, which injects their topological order as an inductive bias via asynchronous message passing.

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Exploring Multi-Agent Reinforcement Learning for Unrelated Parallel Machine Scheduling

2024-11-12 · Maria Zampella, Urtzi Otamendi, Xabier Belaunzaran, Arkaitz Artetxe 외

Scheduling problems pose significant challenges in resource, industry, and operational management. This paper addresses the Unrelated Parallel Machine Scheduling Problem (UPMS) with setup times and resources using a Mult…

Management Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Smart Scheduling based on Deep Reinforcement Learning for Cellular Networks

2021-03-22 · Jian Wang, Chen Xu, Rong Li, Yiqun Ge 외

To improve the system performance towards the Shannon limit, advanced radio resource management mechanisms play a fundamental role. In particular, scheduling should receive much attention, because it allocates radio reso…

Deep Reinforcement Learning Fairness Management reinforcement-learning+3

Scalable Multi-agent Reinforcement Learning for Factory-wide Dynamic Scheduling

2024-09-20 · Jaeyeon Jang, Diego Klabjan, Han Liu, Nital S. Patel 외

Real-time dynamic scheduling is a crucial but notoriously challenging task in modern manufacturing processes due to its high decision complexity. Recently, reinforcement learning (RL) has been gaining attention as an imp…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Graph-Enhanced Deep Reinforcement Learning for Multi-Objective Unrelated Parallel Machine Scheduling

2026-02-08 · Bulent Soykan, Sean Mondesire, Ghaith Rabadi, Grace Bochenek arxiv

The Unrelated Parallel Machine Scheduling Problem (UPMSP) with release dates, setups, and eligibility constraints presents a significant multi-objective challenge. Traditional methods struggle to balance minimizing Total…

Reinforcement Learning Graph Neural Network

Online Task Scheduling for Fog Computing with Multi-Resource Fairness

2020-08-01 · Simeng Bian, Xi Huang, Ziyu Shao

In fog computing systems, one key challenge is online task scheduling, i.e., to decide the resource allocation for tasks that are continuously generated from end devices. The design is challenging because of various unce…

Deep Reinforcement Learning Fairness Scheduling