GPU Cluster Scheduling for Network-Sensitive Deep Learning
We propose a novel GPU-cluster scheduler for distributed DL (DDL) workloads that enables proximity based consolidation of GPU resources based on the DDL jobs' sensitivities to the anticipated communication-network delays. Our scheduler consists of three major components: (i) a classical delay scheduling algorithm to facilitate job placement and consolidation; (ii) a network-sensitive job preemption strategy; and (iii) an "auto-tuner" mechanism to optimize delay timers for effective delay scheduling. Additionally, to enable a cost-effective methodology for large-scale experiments, we develop a data-driven DDL cluster simulation platform. Employing the simulation platform we compare against several state-of-the-art alternatives on real-world workload traces to demonstrate the benefits of our design. Our scheduler can provide improvement of up to 69% in end-to-end Makespan for training all jobs compared to the prevailing consolidation-based scheduling methods, while reducing the average job completion time by up to 83% and minimizing the communication overheads by up to 98% under congested networking conditions.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningGPUSchedulingSimilar Papers 제목 키워드 기반
Synergy: Resource Sensitive DNN Scheduling in Multi-Tenant Clusters
Training Deep Neural Networks (DNNs) is a widely popular workload in both enterprises and cloud data centers. Existing schedulers for DNN training consider GPU as the dominant resource, and allocate other resources such …
CPUGPUSchedulingSensitivityA Codesign of Scheduling and Parallelization for Large Model Training in Heterogeneous Clusters
Joint consideration of scheduling and adaptive parallelism offers great opportunities for improving the training efficiency of large models on heterogeneous GPU clusters. However, integrating adaptive parallelism into a …
GPUSchedulingKant: An Efficient Unified Scheduling System for Large-Scale AI Clusters
As AI cluster sizes continue to expand and the demand for large-language-model (LLM) training and inference workloads grows rapidly, traditional scheduling systems face significant challenges in balancing resource utiliz…
SAGA: Workflow-Atomic Scheduling for AI Agent Inference on GPU Clusters
AI agents execute tens to hundreds of chained LLM calls per task, yet GPU schedulers treat each call as independent, discarding gigabytes of intermediate state between steps and inflating end-to-end latency by 3-8x. We a…
An adaptive Simulated Annealing-based satellite observation scheduling method combined with a dynamic task clustering strategy
Efficient scheduling is of great significance to rationally make use of scarce satellite resources. Task clustering has been demonstrated to realize an effective strategy to improve the efficiency of satellite scheduling…
ClusteringScheduling