Deep Reinforcement Agent for Scheduling in HPC

2021-02-11 · Yuping Fan, Zhiling Lan, Taylor Childers, Paul Rich, William Allcock, Michael E. Papka

Cluster scheduler is crucial in high-performance computing (HPC). It determines when and which user jobs should be allocated to available system resources. Existing cluster scheduling heuristics are developed by human experts based on their experience with specific HPC systems and workloads. However, the increasing complexity of computing systems and the highly dynamic nature of application workloads have placed tremendous burden on manually designed and tuned scheduling heuristics. More aggressive optimization and automation are needed for cluster scheduling in HPC. In this work, we present an automated HPC scheduling agent named DRAS (Deep Reinforcement Agent for Scheduling) by leveraging deep reinforcement learning. DRAS is built on a novel, hierarchical neural network incorporating special HPC scheduling features such as resource reservation and backfilling. A unique training strategy is presented to enable DRAS to rapidly learn the target environment. Once being provided a specific scheduling objective given by system manager, DRAS automatically learns to improve its policy through interaction with the scheduling environment and dynamically adjusts its policy as workload changes. The experiments with different production workloads demonstrate that DRAS outperforms the existing heuristic and optimization approaches by up to 45%.

📄 PDF Abstract BibTeX arXiv:2102.06243

Code (1)

SPEAR-IIT/CQSim 공식 구현

Tasks

Deep Reinforcement LearningScheduling

Methods 이 논문이 사용한 방법론

REINFORCE REINFORCE is a Monte Carlo variant of a policy gradient algorithm in reinforcement learning. The agent collects samples of an episode using its current policy, and uses it to…

DPG Deterministic Policy Gradient, or DPG, is a policy gradient method for reinforcement learning. Instead of the policy function $\pi\left(.\mid{s}\right)$ being modeled as a…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

VMAgent: Scheduling Simulator for Reinforcement Learning

2021-12-09 · Junjie Sheng, Shengliang Cai, Haochuan Cui, Wenhao Li 외

A novel simulator called VMAgent is introduced to help RL researchers better explore new methods, especially for virtual machine scheduling. VMAgent is inspired by practical virtual machine (VM) scheduling tasks and prov…

Cloud Computing reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Exploring Multi-Agent Reinforcement Learning for Unrelated Parallel Machine Scheduling

2024-11-12 · Maria Zampella, Urtzi Otamendi, Xabier Belaunzaran, Arkaitz Artetxe 외

Scheduling problems pose significant challenges in resource, industry, and operational management. This paper addresses the Unrelated Parallel Machine Scheduling Problem (UPMS) with setup times and resources using a Mult…

Management Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Reinforcement Learning Approach for Multi-Agent Flexible Scheduling Problems

2022-10-07 · Hongjian Zhou, Boyang Gu, Chenghao Jin

Scheduling plays an important role in automated production. Its impact can be found in various fields such as the manufacturing industry, the service industry and the technology industry. A scheduling problem (NP-hard) i…

Combinatorial Optimization Decision Making Job Shop Scheduling OpenAI Gym+6

Scalable Multi-agent Reinforcement Learning for Factory-wide Dynamic Scheduling

2024-09-20 · Jaeyeon Jang, Diego Klabjan, Han Liu, Nital S. Patel 외

Real-time dynamic scheduling is a crucial but notoriously challenging task in modern manufacturing processes due to its high decision complexity. Recently, reinforcement learning (RL) has been gaining attention as an imp…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Decentralized scheduling through an adaptive, trading-based multi-agent system

2022-07-05 · Michael Kölle, Lennart Rietdorf, Kyrill Schmid

In multi-agent reinforcement learning systems, the actions of one agent can have a negative impact on the rewards of other agents. One way to combat this problem is to let agents trade their rewards amongst each other. M…