FedRef: Communication-Efficient Bayesian Fine Tuning with Reference Model
Federated learning(FL) is used for distributed scenarios to train artificial intelligence(AI) models while ensuring users' privacy. In federated learning scenario, the server generally never knows about users' data. This type of concept makes the AI training process efficient in terms of data privacy. However, regarding model performance, federated AI models may not sufficiently satisfy AI users' expectations. Furthermore, AI users have a wide range of different needs. It is not easy to satisfy the whole users needs. These types of issues can be addressed through AI model optimization, fine-tuning, or personalization to achieve optimal model performance. To address model optimization challenges, we propose reference model-based federated learning for optimal fine-tuning, which overcomes catastrophic forgetting in each round. This method is derived from Bayesian parameter-efficient transfer learning, which includes an optimal proximal term and enables overcoming the catastrophic forgetting issue in each round by utilizing a reference model that incorporates previous model parameters. As a result, this method achieves both high model performance and low computing cost.
Code (1)
Tasks
Brain Tumor SegmentationFederated LearningModel OptimizationTransfer LearningSimilar Papers 제목 키워드 기반
FedReFT: Federated Representation Fine-Tuning with All-But-Me Aggregation
Parameter-efficient fine-tuning (PEFT) adapts large pre-trained models by updating only a small subset of parameters. Recently, Representation Fine-Tuning (ReFT) has emerged as an effective alternative. ReFT shifts the f…
parameter-efficient fine-tuningArithmetic ReasoningFederated LearningOnline Learning from Strategic Human Feedback in LLM Fine-Tuning
Reinforcement learning from human feedback (RLHF) has become an essential step in fine-tuning large language models (LLMs) to align them with human preferences. However, human labelers are selfish and have diverse prefer…
Cross-Domain Image Captioning with Discriminative Finetuning
Neural captioners are typically trained to mimic human-generated references without optimizing for any specific communication goal, leading to problems such as the generation of vague captions. In this paper, we show tha…
DescriptiveImage CaptioningRL with KL penalties is better viewed as Bayesian inference
Reinforcement learning (RL) is frequently employed in fine-tuning large language models (LMs), such as GPT-3, to penalize them for undesirable features of generated sequences, such as offensiveness, social bias, harmfuln…
Bayesian InferenceLanguage ModellingReinforcement Learning (RL)Variational InferenceEfficient Reinforcement Learning from Human Feedback via Bayesian Preference Inference
Learning from human preferences is a cornerstone of aligning machine learning models with subjective human judgments. Yet, collecting such preference data is often costly and time-consuming, motivating the need for more …
Reinforcement Learning