Federated Customization of Large Models: Approaches, Experiments, and Insights
In this article, we explore federated customization of large models and highlight the key challenges it poses within the federated learning framework. We review several popular large model customization techniques, including full fine-tuning, efficient fine-tuning, prompt engineering, prefix-tuning, knowledge distillation, and retrieval-augmented generation. Then, we discuss how these techniques can be implemented within the federated learning framework. Moreover, we conduct experiments on federated prefix-tuning, which, to the best of our knowledge, is the first trial to apply prefix-tuning in the federated learning setting. The conducted experiments validate its feasibility with performance close to centralized approaches. Further comparison with three other federated customization methods demonstrated its competitive performance, satisfactory efficiency, and consistent robustness.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationFederated LearningPrompt EngineeringSimilar Papers 제목 키워드 기반
Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization
Federated learning (FL) provides a distributed learning framework for multiple participants to collaborate learning without sharing raw data. In many practical FL scenarios, participants have heterogeneous resources due …
Federated LearningPersonalized Federated LearningCOALA: A Practical and Vision-Centric Federated Learning Platform
We present COALA, a vision-centric Federated Learning (FL) platform, and a suite of benchmarks for practical FL scenarios, which we categorize into three levels: task, data, and model. At the task level, COALA extends su…
BenchmarkingContinual LearningFederated Learningobject-detection+2FedSecurity: Benchmarking Attacks and Defenses in Federated Learning and Federated LLMs
This paper introduces FedSecurity, an end-to-end benchmark that serves as a supplementary component of the FedML library for simulating adversarial attacks and corresponding defense mechanisms in Federated Learning (FL).…
BenchmarkingFederated LearningLearnable Sparse Customization in Heterogeneous Edge Computing
To effectively manage and utilize massive distributed data at the network edge, Federated Learning (FL) has emerged as a promising edge computing paradigm across data silos. However, FL still faces two challenges: system…
Edge-computingFederated LearningDreamO: A Unified Framework for Image Customization
Recently, extensive research on image customization (e.g., identity, subject, style, background, etc.) demonstrates strong customization capabilities in large-scale generative models. However, most approaches are designe…