Addressing Data Heterogeneity in Decentralized Learning via Topological Pre-processing
Recently, local peer topology has been shown to influence the overall convergence of decentralized learning (DL) graphs in the presence of data heterogeneity. In this paper, we demonstrate the advantages of constructing a proxy-based locally heterogeneous DL topology to enhance convergence and maintain data privacy. In particular, we propose a novel peer clumping strategy to efficiently cluster peers before arranging them in a final training graph. By showing how locally heterogeneous graphs outperform locally homogeneous graphs of similar size and from the same global data distribution, we present a strong case for topological pre-processing. Moreover, we demonstrate the scalability of our approach by showing how the proposed topological pre-processing overhead remains small in large graphs while the performance gains get even more pronounced. Furthermore, we show the robustness of our approach in the presence of network partitions.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
ColNet: Collaborative Optimization in Decentralized Federated Multi-task Learning Systems
The integration of Federated Learning (FL) and Multi-Task Learning (MTL) has been explored to address client heterogeneity, with Federated Multi-Task Learning (FMTL) treating each client as a distinct task. However, most…
Federated LearningMulti-Task LearningDistribution-Free Fair Federated Learning with Small Samples
As federated learning gains increasing importance in real-world applications due to its capacity for decentralized data training, addressing fairness concerns across demographic groups becomes critically important. Howev…
FairnessFederated LearningFederated Domain Generalization with Label Smoothing and Balanced Decentralized Training
In this paper, we propose a novel approach, Federated Domain Generalization with Label Smoothing and Balanced Decentralized Training (FedSB), to address the challenges of data heterogeneity within a federated learning fr…
Domain GeneralizationFederated LearningDFed-SST: Building Semantic- and Structure-aware Topologies for Decentralized Federated Graph Learning
Decentralized Federated Learning (DFL) has emerged as a robust distributed paradigm that circumvents the single-point-of-failure and communication bottleneck risks of centralized architectures. However, a significant cha…
Federated LearningGraph LearningTowards Effective Federated Multimodal Graph Learning via Navigating Multifaceted Heterogeneity
Multimodal-attributed graphs (MAGs), where nodes carry heterogeneous semantic content across multiple modalities while edges encode relational dependencies, have been widely adopted across diverse domains. Federated mult…
Contrastive LearningGraph Learning