On the Utility of Domain-Adjacent Fine-Tuned Model Ensembles for Few-shot Problems
Large Language Models (LLMs) have been observed to perform well on a wide range of downstream tasks when fine-tuned on domain-specific data. However, such data may not be readily available in many applications, motivating zero-shot or few-shot approaches using domain-adjacent models. While several fine-tuned models for various tasks are available, finding an appropriate domain-adjacent model for a given task is often not straight forward. In this paper, we study DAFT-E, a framework that utilizes an Ensemble of Domain-Adjacent Fine-Tuned Foundation Models for few-shot problems. We show that for zero-shot problems, this ensembling method provides an accuracy performance close to that of the single best model. With few-shot problems, this performance improves further, at which point DEFT-E can outperform any single domain-adjacent model while requiring much less data for domain-specific fine-tuning.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Uncertainty quantification in fine-tuned LLMs using LoRA ensembles
Fine-tuning large language models can improve task specific performance, although a general understanding of what the fine-tuned model has learned, forgotten and how to trust its predictions is still missing. We derive p…
Multiple-choiceUncertainty QuantificationSmall or Large? Zero-Shot or Finetuned? Guiding Language Model Choice for Specialized Applications in Healthcare
This study aims to guide language model selection by investigating: 1) the necessity of finetuning versus zero-shot usage, 2) the benefits of domain-adjacent versus generic pretrained models, 3) the value of further doma…
Language ModelingLanguage ModellingModel SelectionThe University of Edinburgh’s Bengali-Hindi Submissions to the WMT21 News Translation Task
We describe the University of Edinburgh’s Bengali\leftrightarrowHindi constrained systems submitted to the WMT21 News Translation task. We submitted ensembles of Transformer models built with large-scale back-translation…
TranslationEnsembling Finetuned Language Models for Text Classification
Finetuning is a common practice widespread across different communities to adapt pretrained models to particular tasks. Text classification is one of these tasks for which many pretrained models are available. On the oth…
Classificationtext-classificationText ClassificationCOVID-19 Classification Using Staked Ensembles: A Comprehensive Analysis
The issue of COVID-19, increasing with a massive mortality rate. This led to the WHO declaring it as a pandemic. In this situation, it is crucial to perform efficient and fast diagnosis. The reverse transcript polymerase…
Classification