Towards Communication-Efficient and Privacy-Preserving Federated Representation Learning
This paper investigates the feasibility of federated representation learning under the constraints of communication cost and privacy protection. Existing works either conduct annotation-guided local training which requires frequent communication or aggregates the client models via weight averaging which has potential risks of privacy exposure. To tackle the above problems, we first identify that self-supervised contrastive local training is robust against the non-identically distributed data, which provides the feasibility of longer local training and thus reduces the communication cost. Then based on the aforementioned robustness, we propose a novel Federated representation Learning framework with Ensemble Similarity Distillation~(FLESD) that utilizes this robustness. At each round of communication, the server first gathers a fraction of the clients' inferred similarity matrices on a public dataset. Then it ensembles the similarity matrices and train the global model via similarity distillation. We verify the effectiveness of FLESD by a series of empirical experiments and show that, despite stricter constraints, it achieves comparable results under multiple settings on multiple datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningFederated LearningPrivacy PreservingRepresentation LearningSimilar Papers 제목 키워드 기반
Practical Vertical Federated Learning with Unsupervised Representation Learning
As societal concerns on data privacy recently increase, we have witnessed data silos among multiple parties in various applications. Federated learning emerges as a new learning paradigm that enables multiple parties to …
Federated LearningPrivacy PreservingRepresentation LearningVertical Federated LearningProvably Communication-Efficient and Privacy-Preserving Federated Graph Neural Networks
Graph neural networks (GNNs) achieve strong performance on relational data, but real-world graphs are often distributed across organizations that cannot share raw data due to privacy and policy constraints. Existing fede…
FedDCL: a federated data collaboration learning as a hybrid-type privacy-preserving framework based on federated learning and data collaboration
Recently, federated learning has attracted much attention as a privacy-preserving integrated analysis that enables integrated analysis of data held by multiple institutions without sharing raw data. On the other hand, fe…
Federated LearningPrivacy PreservingEfficient-FedRec: Efficient Federated Learning Framework for Privacy-Preserving News Recommendation
News recommendation is critical for personalized news access. Most existing news recommendation methods rely on centralized storage of users' historical news click behavior data, which may lead to privacy concerns and ha…
Federated LearningNews RecommendationPrivacy PreservingPrivacy-preserving Decentralized Aggregation for Federated Learning
Federated learning is a promising framework for learning over decentralized data spanning multiple regions. This approach avoids expensive central training data aggregation cost and can improve privacy because distribute…
Federated Learningimage-classificationImage ClassificationPrivacy Preserving