ConDistFL: Conditional Distillation for Federated Learning from Partially Annotated Data
Developing a generalized segmentation model capable of simultaneously delineating multiple organs and diseases is highly desirable. Federated learning (FL) is a key technology enabling the collaborative development of a model without exchanging training data. However, the limited access to fully annotated training data poses a major challenge to training generalizable models. We propose "ConDistFL", a framework to solve this problem by combining FL with knowledge distillation. Local models can extract the knowledge of unlabeled organs and tumors from partially annotated data from the global model with an adequately designed conditional probability representation. We validate our framework on four distinct partially annotated abdominal CT datasets from the MSD and KiTS19 challenges. The experimental results show that the proposed framework significantly outperforms FedAvg and FedOpt baselines. Moreover, the performance on an external test dataset demonstrates superior generalizability compared to models trained on each dataset separately. Our ablation study suggests that ConDistFL can perform well without frequent aggregation, reducing the communication cost of FL. Our implementation will be available at https://github.com/NVIDIA/NVFlare/tree/dev/research/condist-fl.
Code (1)
Tasks
Federated LearningKnowledge DistillationSimilar Papers 제목 키워드 기반
Federated Learning with Partially Labeled Data: A Conditional Distillation Approach
In medical imaging, developing generalized segmentation models that can handle multiple organs and lesions is crucial. However, the scarcity of fully annotated datasets and strict privacy regulations present significant …
Federated LearningImage SegmentationMedical Image SegmentationPartially Labeled Datasets+2UFPS: A unified framework for partially-annotated federated segmentation in heterogeneous data distribution
Partially supervised segmentation is a label-saving method based on datasets with fractional classes labeled and intersectant. However, it is still far from landing on real-world medical applications due to privacy conce…
Federated LearningSegmentationData exploitation: multi-task learning of object detection and semantic segmentation on partially annotated data
Multi-task partially annotated data where each data point is annotated for only a single task are potentially helpful for data scarcity if a network can leverage the inter-task relationship. In this paper, we study the j…
Knowledge DistillationMulti-Task Learningobject-detectionObject Detection+1FedAUXfdp: Differentially Private One-Shot Federated Distillation
Federated learning suffers in the case of non-iid local datasets, i.e., when the distributions of the clients' data are heterogeneous. One promising approach to this challenge is the recently proposed method FedAUX, an a…
Federated LearningLearning Underrepresented Classes from Decentralized Partially Labeled Medical Images
Using decentralized data for federated training is one promising emerging research direction for alleviating data scarcity in the medical domain. However, in contrast to large-scale fully labeled data commonly seen in ge…
Federated LearningObject RecognitionSelf-Supervised LearningThoracic Disease Classification