On Episodes, Prototypical Networks, and Few-shot Learning
Episodic learning is a popular practice among researchers and practitioners interested in few-shot learning. It consists of organising training in a series of learning problems (or episodes), each divided into a small training and validation subset to mimic the circumstances encountered during evaluation. But is this always necessary? In this paper, we investigate the usefulness of episodic learning in methods which use nonparametric approaches, such as nearest neighbours, at the level of the episode. For these methods, we not only show how the constraints imposed by episodic learning are not necessary, but that they in fact lead to a data-inefficient way of exploiting training batches. We conduct a wide range of ablative experiments with Matching and Prototypical Networks, two of the most popular methods that use nonparametric approaches at the level of the episode. Their "non-episodic" counterparts are considerably simpler, have less hyperparameters, and improve their performance in multiple few-shot classification datasets.
Code (1)
Tasks
Few-Shot LearningSimilar Papers 제목 키워드 기반
Meta-Learning for Semi-Supervised Few-Shot Classification
In few-shot classification, we are interested in learning algorithms that train a classifier from only a handful of labeled examples. Recent progress in few-shot classification has featured meta-learning, in which a para…
General ClassificationMeta-LearningProtAugment: Intent Detection Meta-Learning through Unsupervised Diverse Paraphrasing
Recent research considers few-shot intent detection as a meta-learning problem: the model is learning to learn from a consecutive set of small tasks named episodes. In this work, we propose ProtAugment, a meta-learning a…
DiversityIntent DetectionLanguage ModelingLanguage Modelling+1ProtAugment: Unsupervised diverse short-texts paraphrasing for intent detection meta-learning
Recent research considers few-shot intent detection as a meta-learning problem: the model is learning to learn from a consecutive set of small tasks named episodes. In this work, we propose ProtAugment, a meta-learning a…
DiversityIntent DetectionLanguage ModelingLanguage Modelling+1ALPINE: Adaptive Localization for Parameter- and Sample-Efficient Few-Shot Learning
Few-shot learning research is predominantly evaluated on accuracy alone, with limited attention to the parameter and training-sample budgets required to reach that accuracy - a real constraint for practitioners without l…
Few-Shot Image ClassificationFew-Shot LearningRevisiting Prototypical Network for Cross Domain Few-Shot Learning
Prototypical Network is a popular few-shot solver that aims at establishing a feature metric generalizable to novel few-shot classification (FSC) tasks using deep neural networks. However, its performance drops drama…
Cross-Domain Few-Shotcross-domain few-shot learningFew-Shot LearningKnowledge Distillation