Few-Shot Learning with Siamese Networks and Label Tuning
We study the problem of building text classifiers with little or no training data, commonly known as zero and few-shot text classification. In recent years, an approach based on neural textual entailment models has been found to give strong results on a diverse range of tasks. In this work, we show that with proper pre-training, Siamese networks that embed texts and labels are a competitive alternative. These models allow for a large reduction in inference cost: constant in the number of labels rather than linear. Furthermore, we introduce label tuning, a simple and computationally efficient approach that allows to adapt the models in a few-shot setup by only changing the label embeddings. While giving lower performance than model fine-tuning, this approach has the architectural advantage that a single encoder can be shared by many different tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot LearningFew-Shot Text ClassificationNatural Language Inferencetext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Few-Shot Learning with Siamese Networks and Label Tuning
We study the problem of building text classifiers with little or no training data, commonly known as zero and few-shot text classification. In recent years, an approach based on neural textual entailment models has been …
Few-Shot LearningFew-Shot Text ClassificationNatural Language Inferencetext-classification+1MergedNET: A simple approach for one-shot learning in siamese networks based on similarity layers
Classifiers trained on disjointed classes with few labelled data points are used in one-shot learning to identify visual concepts from other classes. Recently, Siamese networks and similarity layers have been used to sol…
Few-Shot Image ClassificationFine-Grained Image Classificationimage-classificationImage Classification+2Few-shot Learning in Emotion Recognition of Spontaneous Speech Using a Siamese Neural Network with Adaptive Sample Pair Formation
Speech-based machine learning (ML) has been heralded as a promising solution for tracking prosodic and spectrotemporal patterns in real-life that are indicative of emotional changes, providing a valuable window into one'…
Emotion RecognitionFew-Shot LearningMetric LearningOne-Vote Veto: Semi-Supervised Learning for Low-Shot Glaucoma Diagnosis
Convolutional neural networks (CNNs) are a promising technique for automated glaucoma diagnosis from images of the fundus, and these images are routinely acquired as part of an ophthalmic exam. Nevertheless, CNNs typical…
image-classificationImage ClassificationEfficient Few-Shot Learning Without Prompts
Recent few-shot methods, such as parameter-efficient fine-tuning (PEFT) and pattern exploiting training (PET), have achieved impressive results in label-scarce settings. However, they are difficult to employ since they a…
Few-Shot LearningFew-Shot Text Classificationparameter-efficient fine-tuningSentence