Handling Spatial-Temporal Data Heterogeneity for Federated Continual Learning via Tail Anchor
Federated Continual Learning (FCL) allows each client to continually update its knowledge from task streams, enhancing the applicability of federated learning in real-world scenarios. However, FCL needs to address not only spatial data heterogeneity between clients but also temporal data heterogeneity between tasks. In this paper, empirical experiments demonstrate that such input-level heterogeneity significantly affects the model's internal parameters and outputs, leading to severe spatial-temporal catastrophic forgetting of local and previous knowledge. To this end, we propose Federated Tail Anchor (FedTA) to mix trainable Tail Anchor with the frozen output features to adjust their position in the feature space, thereby overcoming parameter-forgetting and output-forgetting. Three novel components are also included: Input Enhancement for improving the performance of pre-trained models on downstream tasks; Selective Input Knowledge Fusion for fusion of heterogeneous local knowledge on the server; and Best Global Prototype Selection for finding the best anchor point for each class in the feature space. Extensive experiments demonstrate that FedTA not only outperforms existing FCL methods but also effectively preserves the relative positions of features.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningFederated LearningPrototype SelectionSimilar Papers 제목 키워드 기반
Addressing Spatial-Temporal Data Heterogeneity in Federated Continual Learning via Tail Anchor
Federated continual learning (FCL) allows each client to continually update its knowledge from task streams, enhancing the applicability of federated learning in real-world scenarios. However, FCL needs to address not on…
Continual LearningFederated LearningPrototype SelectionPersonalized Federated Learning for Spatio-Temporal Forecasting: A Dual Semantic Alignment-Based Contrastive Approach
The existing federated learning (FL) methods for spatio-temporal forecasting fail to capture the inherent spatio-temporal heterogeneity, which calls for personalized FL (PFL) methods to model the spatio-temporally varian…
Contrastive LearningFederated LearningPersonalized Federated LearningSemantic Similarity+2Federated Learning in Temporal Heterogeneity
In this work, we explored federated learning in temporal heterogeneity across clients. We observed that global model obtained by \texttt{FedAvg} trained with fixed-length sequences shows faster convergence than varying-l…
Federated LearningAFCL: Analytic Federated Continual Learning for Spatio-Temporal Invariance of Non-IID Data
Federated Continual Learning (FCL) enables distributed clients to collaboratively train a global model from online task streams in dynamic real-world scenarios. However, existing FCL methods face challenges of both spati…
Continual LearningAn Experimental Study of Data Heterogeneity in Federated Learning Methods for Medical Imaging
Federated learning enables multiple institutions to collaboratively train machine learning models on their local data in a privacy-preserving way. However, its distributed nature often leads to significant heterogeneity …
Federated LearningPrivacy Preserving