Information Entropy-Based Scheduling for Communication-Efficient Decentralized Learning
This paper addresses decentralized stochastic gradient descent (D-SGD) over resource-constrained networks by introducing node-based and link-based scheduling strategies to enhance communication efficiency. In each iteration of the D-SGD algorithm, only a few disjoint subsets of nodes or links are randomly activated, subject to a given communication cost constraint. We propose a novel importance metric based on information entropy to determine node and link scheduling probabilities. We validate the effectiveness of our approach through extensive simulations, comparing it against state-of-the-art methods, including betweenness centrality (BC) for node scheduling and \textit{MATCHA} for link scheduling. The results show that our method consistently outperforms the BC-based method in the node scheduling case, achieving faster convergence with up to 60\% lower communication budgets. At higher communication budgets (above 60\%), our method maintains comparable or superior performance. In the link scheduling case, our method delivers results that are superior to or on par with those of \textit{MATCHA}.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Scheduling and Communication Schemes for Decentralized Federated Learning
Federated learning (FL) is a distributed machine learning paradigm in which a large number of clients coordinate with a central server to learn a model without sharing their own training data. One central server is not e…
Federated LearningSchedulingConcepts and Algorithms for Agent-based Decentralized and Integrated Scheduling of Production and Auxiliary Processes
Individualized products and shorter product life cycles have driven companies to rethink traditional mass production. New concepts like Industry 4.0 foster the advent of decentralized production control and distribution …
SchedulingFaster Convergence with Less Communication: Broadcast-Based Subgraph Sampling for Decentralized Learning over Wireless Networks
Consensus-based decentralized stochastic gradient descent (D-SGD) is a widely adopted algorithm for decentralized training of machine learning models across networked agents. A crucial part of D-SGD is the consensus-base…
SchedulingLoad Balancing in Federated Learning
Federated Learning (FL) is a decentralized machine learning framework that enables learning from data distributed across multiple remote devices, enhancing communication efficiency and data privacy. Due to limited commun…
Decision MakingFairnessFederated LearningSchedulingLoad-Aware Training Scheduling for Model Circulation-based Decentralized Federated Learning
This paper proposes Load-aware Tram-FL, an extension of Tram-FL that introduces a training scheduling mechanism to minimize total training time in decentralized federated learning by accounting for both computational and…
Federated Learningglobal-optimizationScheduling