Gradual Fine-Tuning for Low-Resource Domain Adaptation
Fine-tuning is known to improve NLP models by adapting an initial model trained on more plentiful but less domain-salient examples to data in a target domain. Such domain adaptation is typically done using one stage of fine-tuning. We demonstrate that gradually fine-tuning in a multi-stage process can yield substantial further gains and can be applied without modifying the model or learning objective.
Code (2)
Tasks
Domain AdaptationSimilar Papers 제목 키워드 기반
SemiAdapt and SemiLoRA: Efficient Domain Adaptation for Transformer-based Low-Resource Language Translation with a Case Study on Irish
Fine-tuning is widely used to tailor large language models for specific tasks such as neural machine translation (NMT). However, leveraging transfer learning is computationally expensive when fine-tuning large multilingu…
parameter-efficient fine-tuningMachine TranslationTransfer LearningDomain AdaptationLow-Resource Domain Adaptation for Speech LLMs via Text-Only Fine-Tuning
Recent advances in automatic speech recognition (ASR) have combined speech encoders with large language models (LLMs) through projection, forming Speech LLMs with strong performance. However, adapting them to new domains…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Domain Adaptationspeech-recognition+1A Lightweight Multi-Expert Generative Language Model System for Engineering Information and Knowledge Extraction
Despite recent advancements in domain adaptation techniques for large language models, these methods remain computationally intensive, and the resulting models can still exhibit hallucination issues. Most existing adapta…
Domain AdaptationHallucinationLanguage ModelingLanguage Modelling+1Gradual Fine-Tuning with Graph Routing for Multi-Source Unsupervised Domain Adaptation
Multi-source unsupervised domain adaptation aims to leverage labeled data from multiple source domains for training a machine learning model to generalize well on a target domain without labels. Source domain selection p…
Domain AdaptationMulti-Source Unsupervised Domain AdaptationNatural Language InferenceSentiment Analysis+1A Unified Speaker Adaptation Approach for ASR
Transformer models have been used in automatic speech recognition (ASR) successfully and yields state-of-the-art results. However, its performance is still affected by speaker mismatch between training and test data. Fur…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech Recognition