Federated Continual Learning with Weighted Inter-client Transfer
There has been a surge of interest in continual learning and federated learning, both of which are important in deep neural networks in real-world scenarios. Yet little research has been done regarding the scenario where each client learns on a sequence of tasks from a private local data stream. This problem of federated continual learning poses new challenges to continual learning, such as utilizing knowledge from other clients, while preventing interference from irrelevant knowledge. To resolve these issues, we propose a novel federated continual learning framework, Federated Weighted Inter-client Transfer (FedWeIT), which decomposes the network weights into global federated parameters and sparse task-specific parameters, and each client receives selective knowledge from other clients by taking a weighted combination of their task-specific parameters. FedWeIT minimizes interference between incompatible tasks, and also allows positive knowledge transfer across clients during learning. We validate our FedWeIT against existing federated learning and continual learning methods under varying degrees of task similarity across clients, and our model significantly outperforms them with a large reduction in the communication cost. Code is available at https://github.com/wyjeong/FedWeIT
Code (1)
Tasks
Continual LearningFederated LearningTransfer LearningSimilar Papers 제목 키워드 기반
Federated Continual Learning for Text Classification via Selective Inter-client Transfer
In this work, we combine the two paradigms: Federated Learning (FL) and Continual Learning (CL) for text classification task in cloud-edge continuum. The objective of Federated Continual Learning (FCL) is to improve deep…
Continual LearningFederated Learningtext-classificationText Classification+1FedProK: Trustworthy Federated Class-Incremental Learning via Prototypical Feature Knowledge Transfer
Federated Class-Incremental Learning (FCIL) focuses on continually transferring the previous knowledge to learn new classes in dynamic Federated Learning (FL). However, existing methods do not consider the trustworthines…
class-incremental learningClass Incremental LearningFederated LearningIncremental Learning+1Addressing Client Drift in Federated Continual Learning with Adaptive Optimization
Federated learning has been extensively studied and is the prevalent method for privacy-preserving distributed learning in edge devices. Correspondingly, continual learning is an emerging field targeted towards learning …
Continual LearningFederated LearningPrivacy PreservingFedKNOW: Federated Continual Learning with Signature Task Knowledge Integration at Edge
Deep Neural Networks (DNNs) have been ubiquitously adopted in internet of things and are becoming an integral of our daily life. When tackling the evolving learning tasks in real world, such as classifying different type…
Continual LearningTransfer LearningMasked Autoencoders are Efficient Continual Federated Learners
Machine learning is typically framed from a perspective of i.i.d., and more importantly, isolated data. In parts, federated learning lifts this assumption, as it sets out to solve the real-world challenge of collaborativ…
Continual LearningFederated LearningTransfer Learning