Adaptive Scheduling for Machine Learning Tasks over Networks
A key functionality of emerging connected autonomous systems such as smart transportation systems, smart cities, and the industrial Internet-of-Things, is the ability to process and learn from data collected at different physical locations. This is increasingly attracting attention under the terms of distributed learning and federated learning. However, in this setup data transfer takes place over communication resources that are shared among many users and tasks or subject to capacity constraints. This paper examines algorithms for efficiently allocating resources to linear regression tasks by exploiting the informativeness of the data. The algorithms developed enable adaptive scheduling of learning tasks with reliable performance guarantees.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningFederated LearningInformativenessregressionSchedulingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Enhancing Cluster Scheduling in HPC: A Continuous Transfer Learning for Real-Time Optimization
This study presents a machine learning-assisted approach to optimize task scheduling in cluster systems, focusing on node-affinity constraints. Traditional schedulers like Kubernetes struggle with real-time adaptability,…
Transfer LearningAdaptive Scheduling for Multi-Task Learning
To train neural machine translation models simultaneously on multiple tasks (languages), it is common to sample each task uniformly or in proportion to dataset sizes. As these methods offer little control over performanc…
Machine TranslationMulti-Task LearningSchedulingTranslationMeta-Learning with Neural Bandit Scheduler
Meta-learning has been proven an effective learning paradigm for training machine learning models with good generalization ability. Apart from the common practice of uniformly sampling the meta-training tasks, existing m…
Buffer Pool Aware Query Scheduling via Deep Reinforcement Learning
In this extended abstract, we propose a new technique for query scheduling with the explicit goal of reducing disk reads and thus implicitly increasing query performance. We introduce SmartQueue, a learned scheduler that…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1Flexible Job Shop Scheduling via Dual Attention Network Based Reinforcement Learning
Flexible manufacturing has given rise to complex scheduling problems such as the flexible job shop scheduling problem (FJSP). In FJSP, operations can be processed on multiple machines, leading to intricate relationships …
Decision MakingDeep Reinforcement LearningJob Shop Schedulingreinforcement-learning+1