Multi-Task Federated Learning for Personalised Deep Neural Networks in Edge Computing
Federated Learning (FL) is an emerging approach for collaboratively training Deep Neural Networks (DNNs) on mobile devices, without private user data leaving the devices. Previous works have shown that non-Independent and Identically Distributed (non-IID) user data harms the convergence speed of the FL algorithms. Furthermore, most existing work on FL measures global-model accuracy, but in many cases, such as user content-recommendation, improving individual User model Accuracy (UA) is the real objective. To address these issues, we propose a Multi-Task FL (MTFL) algorithm that introduces non-federated Batch-Normalization (BN) layers into the federated DNN. MTFL benefits UA and convergence speed by allowing users to train models personalised to their own data. MTFL is compatible with popular iterative FL optimisation algorithms such as Federated Averaging (FedAvg), and we show empirically that a distributed form of Adam optimisation (FedAvg-Adam) benefits convergence speed even further when used as the optimisation strategy within MTFL. Experiments using MNIST and CIFAR10 demonstrate that MTFL is able to significantly reduce the number of rounds required to reach a target UA, by up to $5\times$ when using existing FL optimisation strategies, and with a further $3\times$ improvement when using FedAvg-Adam. We compare MTFL to competing personalised FL algorithms, showing that it is able to achieve the best UA for MNIST and CIFAR10 in all considered scenarios. Finally, we evaluate MTFL with FedAvg-Adam on an edge-computing testbed, showing that its convergence and UA benefits outweigh its overhead.
Code (1)
Tasks
Edge-computingFederated LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Multi-task Federated Edge Learning (MtFEEL) in Wireless Networks
Federated Learning (FL) has evolved as a promising technique to handle distributed machine learning across edge devices. A single neural network (NN) that optimises a global objective is generally learned in most work in…
Federated LearningA novel parameter decoupling approach of personalized federated learning for image analysis
Given the importance of privacy protection in databases and other institutions, federated learning (FL) is used to benefit training machine learning models based on these decentralised and private data so as to address t…
Federated LearningPersonalized Federated LearningFedPop: A Bayesian Approach for Personalised Federated Learning
Personalised federated learning (FL) aims at collaboratively learning a machine learning model taylored for each client. Albeit promising advances have been made in this direction, most of existing approaches works do no…
Federated LearningUncertainty QuantificationFedCARE: A Multi-Objective Personalised Federated Learning Framework for Smart Healthcare
Federated Learning (FL) enables collaborative model training across distributed healthcare institutions without centralising sensitive patient data. However, real-world healthcare federations are often characterised not …
Federated LearningPersonalised Federated Learning On Heterogeneous Feature Spaces
Most personalised federated learning (FL) approaches assume that raw data of all clients are defined in a common subspace i.e. all clients store their data according to the same schema. For real-world applications, this …
Federated Learning