DRAS-CQSim: A Reinforcement Learning based Framework for HPC Cluster Scheduling
For decades, system administrators have been striving to design and tune cluster scheduling policies to improve the performance of high performance computing (HPC) systems. However, the increasingly complex HPC systems combined with highly diverse workloads make such manual process challenging, time-consuming, and error-prone. We present a reinforcement learning based HPC scheduling framework named DRAS-CQSim to automatically learn optimal scheduling policy. DRAS-CQSim encapsulates simulation environments, agents, hyperparameter tuning options, and different reinforcement learning algorithms, which allows the system administrators to quickly obtain customized scheduling policies.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningReinforcement Learning (RL)SchedulingSimilar Papers 제목 키워드 기반
Deep Reinforcement Agent for Scheduling in HPC
Cluster scheduler is crucial in high-performance computing (HPC). It determines when and which user jobs should be allocated to available system resources. Existing cluster scheduling heuristics are developed by human ex…
Deep Reinforcement LearningSchedulingNetwork Contention-Aware Cluster Scheduling with Reinforcement Learning
With continuous advances in deep learning, distributed training is becoming common in GPU clusters. Specifically, for emerging workloads with diverse amounts, ratios, and patterns of communication, we observe that networ…
GPUreinforcement-learningReinforcement LearningSchedulingLearning at the Right Pace: Adaptive Data Scheduling Improves LLM Reinforcement Learning
Large Language Models (LLMs) achieve remarkable reasoning capabilities through reinforcement learning (RL) post-training. However, existing RL post-training commonly relies on uniform data sampling, which ignores the sem…
Reinforcement LearningEnergy-aware Scheduling of Jobs in Heterogeneous Cluster Systems Using Deep Reinforcement Learning
Energy consumption is one of the most critical concerns in designing computing devices, ranging from portable embedded systems to computer cluster systems. Furthermore, in the past decade, cluster systems have increasing…
Deep Reinforcement LearningManagementReinforcement LearningReinforcement Learning (RL)+1Scalable Reinforcement Learning for Virtual Machine Scheduling
Recent advancements in reinforcement learning (RL) have shown promise for optimizing virtual machine scheduling (VMS) in small-scale clusters. The utilization of RL to large-scale cloud computing scenarios remains notabl…
Cloud Computingreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1