CAFEDistill: Learning Personalized and Dynamic Models through Federated Early-Exit Network Distillation
Personalized Federated Learning (PFL) enables collaboratively model training on decentralized, heterogeneous data while tailoring them to each client's unique distribution. However, existing PFL methods produce static models with a fixed tradeoff between accuracy and efficiency, limiting their applicability in environments where inference requirements vary with contexts and resource availability. Early-exit networks (EENs) offer adaptive inference by attaching intermediate classifiers. Yet integrating them into PFL is challenging due to client-wise heterogeneity and depth-wise interference arising from conflicting exit objectives. Prior studies fail to resolve both conflicts simultaneously, leading to suboptimal performance. In this paper, we propose CAFEDistill, a Conflict-Aware Federated Exit Distillation framework that jointly addresses these conflicts and extends PFL to early-exit networks. Through a progressive, depth-prioritized student coordination mechanism, CAFEDistill mitigates interference among shallow and deep exits while allowing effective personalized knowledge transfer across clients. Furthermore, it reduces communication overhead via a client-decoupled formulation. Extensive evaluations show that CAFEDistill outperforms the state-of-the-arts, achieving higher accuracy and reducing inference costs by 30.79%-46.86%.
Code (0)
등록된 구현이 없습니다.
Tasks
Personalized Federated LearningSimilar Papers 제목 키워드 기반
UPFL: Unsupervised Personalized Federated Learning towards New Clients
Personalized federated learning has gained significant attention as a promising approach to address the challenge of data heterogeneity. In this paper, we address a relatively unexplored problem in federated learning. Wh…
Federated LearningKnowledge DistillationPersonalized Federated LearningDA-PFL: Dynamic Affinity Aggregation for Personalized Federated Learning
Personalized federated learning becomes a hot research topic that can learn a personalized learning model for each client. Existing personalized federated learning models prefer to aggregate similar clients with similar …
Federated LearningPersonalized Federated LearningDifferential Privacy Personalized Federated Learning Based on Dynamically Sparsified Client Updates
Personalized federated learning is extensively utilized in scenarios characterized by data heterogeneity, facilitating more efficient and automated local training on data-owning terminals. This includes the automated sel…
Federated LearningPersonalized Federated LearningFlow: Per-Instance Personalized Federated Learning Through Dynamic Routing
Personalization in Federated Learning (FL) aims to modify a collaboratively trained global model according to each client. Current approaches to personalization in FL are at a coarse granularity, i.e. all the input insta…
Federated LearningPersonalized Federated LearningpMixFed: Efficient Personalized Federated Learning through Adaptive Layer-Wise Mixup
Traditional Federated Learning (FL) methods encounter significant challenges when dealing with heterogeneous data and providing personalized solutions for non-IID scenarios. Personalized Federated Learning (PFL) approach…
Federated LearningPersonalized Federated Learning