Duration-Informed Workload Scheduler
High-performance computing systems are complex machines whose behaviour is governed by the correct functioning of its many subsystems. Among these, the workload scheduler has a crucial impact on the timely execution of the jobs continuously submitted to the computing resources. Making high-quality scheduling decisions is contingent on knowing the duration of submitted jobs before their execution--a non-trivial task for users that can be tackled with Machine Learning. In this work, we devise a workload scheduler enhanced with a duration prediction module built via Machine Learning. We evaluate its effectiveness and show its performance using workload traces from a Tier-0 supercomputer, demonstrating a decrease in mean waiting time across all jobs of around 11%. Lower waiting times are directly connected to better quality of service from the users' point of view and higher turnaround from the system's perspective.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Workload Schedulers -- Genesis, Algorithms and Differences
This paper presents a novel approach to categorization of modern workload schedulers. We provide descriptions of three classes of schedulers: Operating Systems Process Schedulers, Cluster Systems Jobs Schedulers and Big …
Deep Learning Workload Scheduling in GPU Datacenters: Taxonomy, Challenges and Vision
Deep learning (DL) shows its prosperity in a wide variety of fields. The development of a DL model is a time-consuming and resource-intensive procedure. Hence, dedicated GPU accelerators have been collectively constructe…
GPUSchedulingMixture-of-Schedulers: An Adaptive Scheduling Agent as a Learned Router for Expert Policies
Modern operating system schedulers employ a single, static policy, which struggles to deliver optimal performance across the diverse and dynamic workloads of contemporary systems. This "one-policy-fits-all" approach lead…
DREAM: A Dynamic Scheduler for Dynamic Real-time Multi-model ML Workloads
Emerging real-time multi-model ML (RTMM) workloads such as AR/VR and drone control involve dynamic behaviors in various granularity; task, model, and layers within a model. Such dynamic behaviors introduce new challenges…
SchedulingRLScheduler: An Automated HPC Batch Job Scheduler Using Reinforcement Learning
Today high-performance computing (HPC) platforms are still dominated by batch jobs. Accordingly, effective batch job scheduling is crucial to obtain high system efficiency. Existing HPC batch job schedulers typically lev…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Scheduling