VIABLE: Fast Adaptation via Backpropagating Learned Loss
In few-shot learning, typically, the loss function which is applied at test time is the one we are ultimately interested in minimising, such as the mean-squared-error loss for a regression problem. However, given that we have few samples at test time, we argue that the loss function that we are interested in minimising is not necessarily the loss function most suitable for computing gradients in a few-shot setting. We propose VIABLE, a generic meta-learning extension that builds on existing meta-gradient-based methods by learning a differentiable loss function, replacing the pre-defined inner-loop loss function in performing task-specific updates. We show that learning a loss function capable of leveraging relational information between samples reduces underfitting, and significantly improves performance and sample efficiency on a simple regression task. Furthermore, we show VIABLE is scalable by evaluating on the Mini-Imagenet dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot LearningMeta-LearningregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Backpropagating through Fréchet Inception Distance
The Fr\'echet Inception Distance (FID) has been used to evaluate hundreds of generative models. We introduce FastFID, which can efficiently train generative models with FID as a loss function. Using FID as an additional …
A Deep Fusion Model for Domain Adaptation in Phrase-based MT
We present a novel fusion model for domain adaptation in Statistical Machine Translation. Our model is based on the joint source-target neural network Devlin et al., 2014, and is learned by fusing in- and out-domain mode…
Domain AdaptationMachine TranslationTranslationWord EmbeddingsLearning State-Dependent Losses for Inverse Dynamics Learning
Being able to quickly adapt to changes in dynamics is paramount in model-based control for object manipulation tasks. In order to influence fast adaptation of the inverse dynamics model's parameters, data efficiency is c…
Meta-LearningTest-Time Meta-Adaptation with Self-Synthesis
As strong general reasoners, large language models (LLMs) encounter diverse domains and tasks, where the ability to adapt and self-improve at test time is valuable. We introduce MASS, a meta-learning framework that enabl…
Mathematical ReasoningBilevel OptimizationTest-time AdaptationFast Task Adaptation for Few-Shot Learning
Few-shot classification is a challenging task due to the scarcity of training examples for each class. The key lies in generalization of prior knowledge learned from large-scale base classes and fast adaptation of the cl…
Few-Shot Learning