FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and Federated Image Classification
Modern deep learning systems are increasingly deployed in situations such as personalization and federated learning where it is necessary to support i) learning on small amounts of data, and ii) communication efficient distributed training protocols. In this work, we develop FiLM Transfer (FiT) which fulfills these requirements in the image classification setting by combining ideas from transfer learning (fixed pretrained backbones and fine-tuned FiLM adapter layers) and meta-learning (automatically configured Naive Bayes classifiers and episodic training) to yield parameter efficient models with superior classification accuracy at low-shot. The resulting parameter efficiency is key for enabling few-shot learning, inexpensive model updates for personalization, and communication efficient federated learning. We experiment with FiT on a wide range of downstream datasets and show that it achieves better classification accuracy than the leading Big Transfer (BiT) algorithm at low-shot and achieves state-of-the art accuracy on the challenging VTAB-1k benchmark, with fewer than 1% of the updateable parameters. Finally, we demonstrate the parameter efficiency and superior accuracy of FiT in distributed low-shot applications including model personalization and federated learning where model update size is an important performance metric.
Code (1)
Tasks
Federated LearningFew-Shot Learningimage-classificationImage ClassificationMeta-LearningTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
FedBiCross: Personalized One-Shot Federated Learning on Medical Images
Data-free knowledge distillation-based one-shot federated learning (OSFL) trains a model in a single communication round without sharing raw data, making OSFL attractive for privacy-sensitive medical applications. Howeve…
Data-free Knowledge DistillationFederated LearningFine-Tuning Impairs the Balancedness of Foundation Models in Long-tailed Personalized Federated Learning
Personalized federated learning (PFL) with foundation models has emerged as a promising paradigm enabling clients to adapt to heterogeneous data distributions. However, real-world scenarios often face the co-occurrence o…
Personalized Federated LearningPersonalized Quantum Federated Learning for Privacy Image Classification
Quantum federated learning has brought about the improvement of privacy image classification, while the lack of personality of the client model may contribute to the suboptimal of quantum federated learning. A personaliz…
ClassificationFederated Learningimage-classificationImage Classification+1Parameterized Knowledge Transfer for Personalized Federated Learning
In recent years, personalized federated learning (pFL) has attracted increasing attention for its potential in dealing with statistical heterogeneity among clients. However, the state-of-the-art pFL methods rely on model…
Federated LearningPersonalized Federated LearningTransfer LearningPersonalized Federated Learning with Residual Fisher Information for Medical Image Segmentation
Federated learning enables multiple clients (institutions) to collaboratively train machine learning models without sharing their private data. To address the challenge of data heterogeneity across clients, personalized …
Personalized Federated LearningMedical Image Segmentation