Peer-to-Peer Deep Learning for Beyond-5G IoT
We present P2PL, a practical multi-device peer-to-peer deep learning algorithm that, unlike the federated learning paradigm, does not require coordination from edge servers or the cloud. This makes P2PL well-suited for the sheer scale of beyond-5G computing environments like smart cities that otherwise create range, latency, bandwidth, and single point of failure issues for federated approaches. P2PL introduces max norm synchronization to catalyze training, retains on-device deep model training to preserve privacy, and leverages local inter-device communication to implement distributed consensus. Each device iteratively alternates between two phases: 1) on-device learning and 2) peer-to-peer cooperation where they combine model parameters with nearby devices. We empirically show that all participating devices achieve the same test performance attained by federated and centralized training -- even with 100 devices and relaxed singly stochastic consensus weights. We extend these experimental results to settings with diverse network topologies, sparse and intermittent communication, and non-IID data distributions.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningFederated LearningSimilar Papers 제목 키워드 기반
Identification of a Rank-dependent Peer Effect Model
We develop a model that captures peer effect heterogeneity by modeling the endogenous spillover to be linear in ordered peer outcomes. Unlike the canonical linear-in-means model, our approach accounts for the distributio…
modelPeerRank: Autonomous LLM Evaluation Through Web-Grounded, Bias-Controlled Peer Review
Evaluating large language models typically relies on human-authored benchmarks, reference answers, and human or single-model judgments, approaches that scale poorly, become quickly outdated, and mismatch open-world deplo…
NLPeer: A Unified Resource for the Computational Study of Peer Review
Peer review constitutes a core component of scholarly publishing; yet it demands substantial expertise and training, and is susceptible to errors and biases. Various applications of NLP for peer reviewing assistance aim …
Peer Learning: Learning Complex Policies in Groups from Scratch via Action Recommendations
Peer learning is a novel high-level reinforcement learning framework for agents learning in groups. While standard reinforcement learning trains an individual agent in trial-and-error fashion, all on its own, peer learni…
OpenAI Gymreinforcement-learningReinforcement LearningPeer-to-Peer Learning Dynamics of Wide Neural Networks
Peer-to-peer learning is an increasingly popular framework that enables beyond-5G distributed edge devices to collaboratively train deep neural networks in a privacy-preserving manner without the aid of a central server.…
Distributed OptimizationPrivacy Preserving