Semi-Supervised Federated Learning with non-IID Data: Algorithm and System Design
Federated Learning (FL) allows edge devices (or clients) to keep data locally while simultaneously training a shared high-quality global model. However, current research is generally based on an assumption that the training data of local clients have ground-truth. Furthermore, FL faces the challenge of statistical heterogeneity, i.e., the distribution of the client's local training data is non-independent identically distributed (non-IID). In this paper, we present a robust semi-supervised FL system design, where the system aims to solve the problem of data availability and non-IID in FL. In particular, this paper focuses on studying the labels-at-server scenario where there is only a limited amount of labeled data on the server and only unlabeled data on the clients. In our system design, we propose a novel method to tackle the problems, which we refer to as Federated Mixing (FedMix). FedMix improves the naive combination of FL and semi-supervised learning methods and designs parameter decomposition strategies for disjointed learning of labeled, unlabeled data, and global models. To alleviate the non-IID problem, we propose a novel aggregation rule based on the frequency of the client's participation in training, namely the FedFreq aggregation algorithm, which can adjust the weight of the corresponding local model according to this frequency. Extensive evaluations conducted on CIFAR-10 dataset show that the performance of our proposed method is significantly better than those of the current baseline. It is worth noting that our system is robust to different non-IID levels of client data.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningSimilar Papers 제목 키워드 기반
FedParsing: a Semi-Supervised Federated Learning Model on Semantic Parsing
Although many semantic parsing models have been proven to work effectively on "NL-to-SQL", the limitation of annotated datasets remains a great challenge. In many semi-supervised models, while they use unlabeled data to …
Federated LearningSemantic ParsingSemi-supervised Federated Learning for Activity Recognition
Training deep learning models on in-home IoT sensory data is commonly used to recognise human activities. Recently, federated learning systems that use edge devices as clients to support local human activity recognition …
Activity RecognitionData AugmentationFederated LearningHuman Activity RecognitionAddressing Class Variable Imbalance in Federated Semi-supervised Learning
Federated Semi-supervised Learning (FSSL) combines techniques from both fields of federated and semi-supervised learning to improve the accuracy and performance of models in a distributed environment by using a small fra…
Semi-FedSER: Semi-supervised Learning for Speech Emotion Recognition On Federated Learning using Multiview Pseudo-Labeling
Speech Emotion Recognition (SER) application is frequently associated with privacy concerns as it often acquires and transmits speech data at the client-side to remote cloud platforms for further processing. These speech…
Emotion RecognitionFederated LearningSpeech Emotion RecognitionExploiting Unlabeled Data in Smart Cities using Federated Learning
Privacy concerns are considered one of the main challenges in smart cities as sharing sensitive data brings threatening problems to people's lives. Federated learning has emerged as an effective technique to avoid privac…
Federated Learning