Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learning
Federated learning is an important privacy-preserving multi-party learning paradigm, involving collaborative learning with others and local updating on private data. Model heterogeneity and catastrophic forgetting are two crucial challenges, which greatly limit the applicability and generalizability. This paper presents a novel FCCL+, federated correlation and similarity learning with non-target distillation, facilitating the both intra-domain discriminability and inter-domain generalization. For heterogeneity issue, we leverage irrelevant unlabeled public data for communication between the heterogeneous participants. We construct cross-correlation matrix and align instance similarity distribution on both logits and feature levels, which effectively overcomes the communication barrier and improves the generalizable ability. For catastrophic forgetting in local updating stage, FCCL+ introduces Federated Non Target Distillation, which retains inter-domain knowledge while avoiding the optimization conflict issue, fulling distilling privileged inter-domain information through depicting posterior classes relation. Considering that there is no standard benchmark for evaluating existing heterogeneous federated learning under the same setting, we present a comprehensive benchmark with extensive representative methods under four domain shift scenarios, supporting both heterogeneous and homogeneous federated settings. Empirical results demonstrate the superiority of our method and the efficiency of modules on various scenarios.
Code (2)
Tasks
Domain GeneralizationFederated LearningPrivacy PreservingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learn From Others and Be Yourself in Heterogeneous Federated Learning
Federated learning has emerged as an important distributed learning paradigm, which normally involves collaborative updating with others and local updating on private data. However, heterogeneity problem and catastro…
Continual LearningFederated Learningimage-classificationImage Classification+1FedHarmony: Harmonizing Heterogeneous Label Correlations in Federated Multi-Label Learning
Federated Multi-Label Learning is a distributed paradigm where multiple clients possess heterogeneous multi-label data and perform collaborative learning under privacy constraints without sharing raw data. However, model…
Multi-Label LearningFederated Learning for Epileptic Seizure Prediction Across Heterogeneous EEG Datasets
Developing accurate and generalizable epileptic seizure prediction models from electroencephalography (EEG) data across multiple clinical sites is hindered by patient privacy regulations and significant data heterogeneit…
Federated LearningSeizure predictionCCFC++: Enhancing Federated Clustering through Feature Decorrelation
In federated clustering, multiple data-holding clients collaboratively group data without exchanging raw data. This field has seen notable advancements through its marriage with contrastive learning, exemplified by Clust…
ClusteringContrastive LearningFederated Learning of Molecular Properties with Graph Neural Networks in a Heterogeneous Setting
Chemistry research has both high material and computational costs to conduct experiments. Institutions thus consider chemical data to be valuable and there have been few efforts to construct large public datasets for mac…
Federated Learning