Improving Few-Shot Learning through Multi-task Representation Learning Theory
In this paper, we consider the framework of multi-task representation (MTR) learning where the goal is to use source tasks to learn a representation that reduces the sample complexity of solving a target task. We start by reviewing recent advances in MTR theory and show that they can provide novel insights for popular meta-learning algorithms when analyzed within this framework. In particular, we highlight a fundamental difference between gradient-based and metric-based algorithms in practice and put forward a theoretical analysis to explain it. Finally, we use the derived insights to improve the performance of meta-learning methods via a new spectral-based regularization term and confirm its efficiency through experimental studies on few-shot classification benchmarks. To the best of our knowledge, this is the first contribution that puts the most recent learning bounds of MTR theory into practice for the task of few-shot classification.
Code (1)
Tasks
Continual LearningFew-Shot LearningLearning TheoryMeta-LearningRepresentation LearningSimilar Papers 제목 키워드 기반
Understanding Few-Shot Multi-Task Representation Learning Theory
Multi-Task Representation Learning (MTR) is a popular paradigm to learn shared representations from multiple related tasks. It has demonstrated its efficiency for solving different problems, ranging from machine translat…
Few-Shot LearningLearning TheoryMachine TranslationMeta-Learning+4A Generalization Theory for Zero-Shot Prediction
A modern paradigm for generalization in machine learning and AI consists of pre-training a task-agnostic foundation model, generally obtained using self-supervised and multimodal contrastive learning. The resulting repre…
Contrastive LearningMinding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker
Theory of Mind (ToM)$\unicode{x2014}$the ability to reason about the mental states of other people$\unicode{x2014}$is a key element of our social intelligence. Yet, despite their ever more impressive performance, large-s…
Reading ComprehensionUsing Grounded Theory for Agent Behavior Analysis at Scale
Understanding agent behavior requires methods that scale to thousands of trajectories and surface new patterns in long, often unfamiliar tasks where pre-built classifiers fall short. We propose to bring grounded theory i…
Disentangling Representations through Multi-task Learning
Intelligent perception and interaction with the world hinges on internal representations that capture its underlying structure (''disentangled'' or ''abstract'' representations). Disentangled representations serve as wor…
Decision MakingMulti-Task LearningZero-shot Generalization