FedSSC: Shared Supervised-Contrastive Federated Learning

2023-01-14 · Sirui Hu, Ling Feng, Xiaohan Yang, Yongchao Chen

Federated learning is widely used to perform decentralized training of a global model on multiple devices while preserving the data privacy of each device. However, it suffers from heterogeneous local data on each training device which increases the difficulty to reach the same level of accuracy as the centralized training. Supervised Contrastive Learning which outperform cross-entropy tries to minimizes the difference between feature space of points belongs to the same class and pushes away points from different classes. We propose Supervised Contrastive Federated Learning in which devices can share the learned class-wise feature spaces with each other and add the supervised-contrastive learning loss as a regularization term to foster the feature space learning. The loss tries to minimize the cosine similarity distance between the feature map and the averaged feature map from another device in the same class and maximizes the distance between the feature map and that in a different class. This new regularization term when added on top of the moon regularization term is found to outperform the other state-of-the-art regularization terms in solving the heterogeneous data distribution problem.

📄 PDF Abstract BibTeX arXiv:2301.05797

Code (0)

등록된 구현이 없습니다.

Tasks

Contrastive Learning Federated Learning

Methods 이 논문이 사용한 방법론

Contrastive Learning 설명 없음

Federated Contrastive Learning of Graph-Level Representations

2024-11-18 · Xiang Li, Gagan Agrawal, Rajiv Ramnath, Ruoming Jin

Graph-level representations (and clustering/classification based on these representations) are required in a variety of applications. Examples include identifying malicious network traffic, prediction of protein properti…

Clustering Contrastive Learning Federated Learning Traffic Prediction

FedSC: Provable Federated Self-supervised Learning with Spectral Contrastive Objective over Non-i.i.d. Data

2024-05-07 · Shusen Jing, Anlan Yu, Shuai Zhang, Songyang Zhang

Recent efforts have been made to integrate self-supervised learning (SSL) with the framework of federated learning (FL). One unique challenge of federated self-supervised learning (FedSSL) is that the global objective of…

Federated Learning Self-Supervised Learning

Federated Self-Supervised Contrastive Learning and Masked Autoencoder for Dermatological Disease Diagnosis

2022-08-24 · Yawen Wu, Dewen Zeng, Zhepeng Wang, Yi Sheng 외

In dermatological disease diagnosis, the private data collected by mobile dermatology assistants exist on distributed mobile devices of patients. Federated learning (FL) can use decentralized data to train models while k…

Contrastive Learning Federated Learning Self-Supervised Learning

Federated Contrastive Learning for Volumetric Medical Image Segmentation

2022-04-23 · Yawen Wu, Dewen Zeng, Zhepeng Wang, Yiyu Shi 외

Supervised deep learning needs a large amount of labeled data to achieve high performance. However, in medical imaging analysis, each site may only have a limited amount of data and labels, which makes learning ineffecti…

Contrastive Learning Federated Learning Image Segmentation Medical Image Segmentation+3

Federated Contrastive Representation Learning with Feature Fusion and Neighborhood Matching

2021-09-29 · Yawen Wu, Zhepeng Wang, Dewen Zeng, Meng Li 외

Federated learning (FL) enables distributed clients to learn a shared model for prediction while keeping the training data local on each client. However, existing FL requires fully-labeled data for training, which is inc…

Contrastive Learning Federated Learning Representation Learning Self-Supervised Learning