Data-Free Evaluation of User Contributions in Federated Learning
Federated learning (FL) trains a machine learning model on mobile devices in a distributed manner using each device's private data and computing resources. A critical issues is to evaluate individual users' contributions so that (1) users' effort in model training can be compensated with proper incentives and (2) malicious and low-quality users can be detected and removed. The state-of-the-art solutions require a representative test dataset for the evaluation purpose, but such a dataset is often unavailable and hard to synthesize. In this paper, we propose a method called Pairwise Correlated Agreement (PCA) based on the idea of peer prediction to evaluate user contribution in FL without a test dataset. PCA achieves this using the statistical correlation of the model parameters uploaded by users. We then apply PCA to designing (1) a new federated learning algorithm called Fed-PCA, and (2) a new incentive mechanism that guarantees truthfulness. We evaluate the performance of PCA and Fed-PCA using the MNIST dataset and a large industrial product recommendation dataset. The results demonstrate that our Fed-PCA outperforms the canonical FedAvg algorithm and other baseline methods in accuracy, and at the same time, PCA effectively incentivizes users to behave truthfully.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningProduct RecommendationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Knowledge-Free Correlated Agreement for Incentivizing Federated Learning
We introduce Knowledge-Free Correlated Agreement (KFCA) to reward client contributions in federated learning (FL) without relying on ground truth, a public test set, or distribution knowledge. Under categorical reports a…
Federated LearningA Reputation Mechanism Is All You Need: Collaborative Fairness and Adversarial Robustness in Federated Learning
Federated learning (FL) is an emerging practical framework for effective and scalable machine learning among multiple participants, such as end users, organizations and companies. However, most existing FL or distributed…
Adversarial DefenseAdversarial RobustnessAllCollaborative Fairness+2Transparent Contribution Evaluation for Secure Federated Learning on Blockchain
Federated Learning is a promising machine learning paradigm when multiple parties collaborate to build a high-quality machine learning model. Nonetheless, these parties are only willing to participate when given enough i…
BIG-bench Machine LearningFederated LearningFederated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metrics
Federated learning (FL) enables collaborative training of a machine learning (ML) model across multiple parties, facilitating the preservation of users' and institutions' privacy by maintaining data stored locally. Inste…
Federated LearningCentral Server Free Federated Learning over Single-sided Trust Social Networks
Federated learning has become increasingly important for modern machine learning, especially for data privacy-sensitive scenarios. Existing federated learning mostly adopts the central server-based architecture or centra…
Federated Learning