Unsupervised Calibration through Prior Adaptation for Text Classification using Large Language Models
A wide variety of natural language tasks are currently being addressed with large-scale language models (LLMs). These models are usually trained with a very large amount of unsupervised text data and adapted to perform a downstream natural language task using methods like fine-tuning, calibration or in-context learning. In this work, we propose an approach to adapt the prior class distribution to perform text classification tasks without the need for labelled samples and only few in-domain sample queries. The proposed approach treats the LLM as a black box, adding a stage where the model posteriors are calibrated to the task. Results show that these methods outperform the un-adapted model for different number of training shots in the prompt and a previous approach were calibration is performed without using any adaptation data.
Code (1)
Tasks
In-Context Learningtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Uncertainty-Aware (Un)Supervised Few-Shot User Adaptation for On-Device Personalized Human Activity Recognition
Sensor-based Human Activity Recognition (HAR) models often degrade on unseen users due to domain shifts caused by individual movement patterns and sensor placement. Practical wearable HAR systems therefore require person…
Human Activity RecognitionTest Time Adaptation Using Adaptive Quantile Recalibration
Domain adaptation is a key strategy for enhancing the generalizability of deep learning models in real-world scenarios, where test distributions often diverge significantly from the training domain. However, conventional…
Test-time AdaptationDomain AdaptationUMFC: Unsupervised Multi-Domain Feature Calibration for Vision-Language Models
Pre-trained vision-language models (e.g., CLIP) have shown powerful zero-shot transfer capabilities. But they still struggle with domain shifts and typically require labeled data to adapt to downstream tasks, which could…
Test-time AdaptationTransductive LearningPseudoCal: A Source-Free Approach to Unsupervised Uncertainty Calibration in Domain Adaptation
Unsupervised domain adaptation (UDA) has witnessed remarkable advancements in improving the accuracy of models for unlabeled target domains. However, the calibration of predictive uncertainty in the target domain, a cruc…
Domain AdaptationUnsupervised Domain AdaptationAUGCAL: Improving Sim2Real Adaptation by Uncertainty Calibration on Augmented Synthetic Images
Synthetic data (SIM) drawn from simulators have emerged as a popular alternative for training models where acquiring annotated real-world images is difficult. However, transferring models trained on synthetic images to r…
Domain AdaptationUnsupervised Domain Adaptation