Prototypical Fine-tuning: Towards Robust Performance Under Varying Data Sizes
In this paper, we move towards combining large parametric models with non-parametric prototypical networks. We propose prototypical fine-tuning, a novel prototypical framework for fine-tuning pretrained language models (LM), which automatically learns a bias to improve predictive performance for varying data sizes, especially low-resource settings. Our prototypical fine-tuning approach can automatically adjust the model capacity according to the number of data points and the model's inherent attributes. Moreover, we propose four principles for effective prototype fine-tuning towards the optimal solution. Experimental results across various datasets show that our work achieves significant performance improvements under various low-resource settings, as well as comparable and usually better performances in high-resource scenarios.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Selecting Fine-Tuning Examples by Quizzing VLMs
A challenge in fine-tuning text-to-image diffusion models for specific topics is to select good examples. Fine-tuning from image sets of varying quality, such as Wikipedia Commons, will often produce poor output. However…
parameter-efficient fine-tuningVisual ReasoningP3T: Prototypical Point-level Prompt Tuning with Enhanced Generalization for 3D Vision-Language Models
With the rise of pre-trained models in the 3D point cloud domain for a wide range of real-world applications, adapting them to downstream tasks has become increasingly important. However, conventional full fine-tuning me…
Few-Shot LearningPrototypical Variational Autoencoder for 3D Few-shot Object Detection
Few-Shot 3D Point Cloud Object Detection (FS3D) is a challenging task, aiming to detect 3D objects of novel classes using only limited annotated samples for training. Considering that the detection performance highly rel…
Prototypical Reward Network for Data-Efficient RLHF
The reward model for Reinforcement Learning from Human Feedback (RLHF) has proven effective in fine-tuning Large Language Models (LLMs). Notably, collecting human feedback for RLHF can be resource-intensive and lead to s…
Prototypical Contrastive Learning-based CLIP Fine-tuning for Object Re-identification
This work aims to adapt large-scale pre-trained vision-language models, such as contrastive language-image pretraining (CLIP), to enhance the performance of object reidentification (Re-ID) across various supervision sett…
Contrastive LearningPerson Re-IdentificationPrompt LearningUnsupervised Person Re-Identification+1