TonY: An Orchestrator for Distributed Machine Learning Jobs
Training machine learning (ML) models on large datasets requires considerable computing power. To speed up training, it is typical to distribute training across several machines, often with specialized hardware like GPUs or TPUs. Managing a distributed training job is complex and requires dealing with resource contention, distributed configurations, monitoring, and fault tolerance. In this paper, we describe TonY, an open-source orchestrator for distributed ML jobs built at LinkedIn to address these challenges.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Motivating Learners in Multi-Orchestrator Mobile Edge Learning: A Stackelberg Game Approach
Mobile Edge Learning (MEL) is a learning paradigm that enables distributed training of Machine Learning models over heterogeneous edge devices (e.g., IoT devices). Multi-orchestrator MEL refers to the coexistence of mult…
SURREAL-System: Fully-Integrated Stack for Distributed Deep Reinforcement Learning
We present an overview of SURREAL-System, a reproducible, flexible, and scalable framework for distributed reinforcement learning (RL). The framework consists of a stack of four layers: Provisioner, Orchestrator, Protoco…
CPUDeep Reinforcement LearningOpenAI Gymreinforcement-learning+3Venn: Resource Management for Collaborative Learning Jobs
In recent years, collaborative learning (CL) has emerged as a promising approach for machine learning (ML) and data science across distributed edge devices. As the deployment of CL jobs increases, they inevitably contend…
Federated LearningManagementSchedulingEnergy-Efficient Multi-Orchestrator Mobile Edge Learning
Mobile Edge Learning (MEL) is a collaborative learning paradigm that features distributed training of Machine Learning (ML) models over edge devices (e.g., IoT devices). In MEL, possible coexistence of multiple learning …
Partitioning Distributed Compute Jobs with Reinforcement Learning and Graph Neural Networks
From natural language processing to genome sequencing, large-scale machine learning models are bringing advances to a broad range of fields. Many of these models are too large to be trained on a single machine, and inste…
BlockingGraph Neural NetworkManagementreinforcement-learning+1