Evaluation Framework For Large-scale Federated Learning
Federated learning is proposed as a machine learning setting to enable distributed edge devices, such as mobile phones, to collaboratively learn a shared prediction model while keeping all the training data on device, which can not only take full advantage of data distributed across millions of nodes to train a good model but also protect data privacy. However, learning in scenario above poses new challenges. In fact, data across a massive number of unreliable devices is likely to be non-IID (identically and independently distributed), which may make the performance of models trained by federated learning unstable. In this paper, we introduce a framework designed for large-scale federated learning which consists of approaches to generating dataset and modular evaluation framework. Firstly, we construct a suite of open-source non-IID datasets by providing three respects including covariate shift, prior probability shift, and concept shift, which are grounded in real-world assumptions. In addition, we design several rigorous evaluation metrics including the number of network nodes, the size of datasets, the number of communication rounds and communication resources etc. Finally, we present an open-source benchmark for large-scale federated learning research.
Code (1)
Tasks
Federated LearningSimilar Papers 제목 키워드 기반
Federated Evaluation and Tuning for On-Device Personalization: System Design & Applications
We describe the design of our federated task processing system. Originally, the system was created to support two specific federated tasks: evaluation and tuning of on-device ML systems, primarily for the purpose of pers…
Federated LearningLLMs meet Federated Learning for Scalable and Secure IoT Management
The rapid expansion of IoT ecosystems introduces severe challenges in scalability, security, and real-time decision-making. Traditional centralized architectures struggle with latency, privacy concerns, and excessive res…
Computational EfficiencyDecision MakingFederated LearningLanguage Modeling+3WW-FL: Secure and Private Large-Scale Federated Learning
Federated learning (FL) is an efficient approach for large-scale distributed machine learning that promises data privacy by keeping training data on client devices. However, recent research has uncovered vulnerabilities …
Data PoisoningFederated LearningModel PoisoningAsynchronous Upper Confidence Bound Algorithms for Federated Linear Bandits
Linear contextual bandit is a popular online learning problem. It has been mostly studied in centralized learning settings. With the surging demand of large-scale decentralized model learning, e.g., federated learning, h…
Federated LearningFederated Adversarial Learning for Robust Autonomous Landing Runway Detection
As the development of deep learning techniques in autonomous landing systems continues to grow, one of the major challenges is trust and security in the face of possible adversarial attacks. In this paper, we propose a f…
Federated LearningLane Detectionparameter-efficient fine-tuning