Understanding Few-Shot Multi-Task Representation Learning Theory

2022-01-17 · ICLR Track Blog 2022 5 · Anonymous

Multi-Task Representation Learning (MTR) is a popular paradigm to learn shared representations from multiple related tasks. It has demonstrated its efficiency for solving different problems, ranging from machine translation for natural language processing to object detection in computer vision. On the other hand, Few-Shot Learning is a recent problem that seeks to mimic the human capability to quickly learn how to solve a target task with little supervision. For this topic, researchers have turned to meta-learning that learns to learn a new task by training a model on a lot of small tasks. As meta-learning still suffers from a lack of theoretical understanding for its success in few-shot tasks, an intuitively appealing approach would be to bridge the gap between it and multi-task learning to better understand the former using the results established for the latter. In this post, we dive into a recent ICLR 2021 paper by Du et al. that demonstrated novel learning bounds for multi-task learning in the few-shot setting and go beyond it by establishing the connections that allow to better understand the inner workings of meta-learning algorithms as well.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Few-Shot LearningLearning TheoryMachine Translation Meta-Learning Multi-Task Learning object-detection Object Detection Representation Learning

Improving Few-Shot Learning through Multi-task Representation Learning Theory

2020-10-05 · Quentin Bouniot, Ievgen Redko, Romaric Audigier, Angélique Loesch 외

In this paper, we consider the framework of multi-task representation (MTR) learning where the goal is to use source tasks to learn a representation that reduces the sample complexity of solving a target task. We start b…

Continual Learning Few-Shot Learning Learning Theory Meta-Learning+1

Few-Shot Character Understanding in Movies as an Assessment to Meta-Learning of Theory-of-Mind

2022-11-09 · Mo Yu, Qiujing Wang, Shunchi Zhang, Yisi Sang 외

When reading a story, humans can quickly understand new fictional characters with a few observations, mainly by drawing analogies to fictional and real people they already know. This reflects the few-shot and meta-learni…

Meta-Learning Metric Learning

Using Grounded Theory for Agent Behavior Analysis at Scale

2026-08-31 · Zhuoran Lu, Yangyang Yu, Zhuoyan Li, Yibo Meng 외 arxiv

Understanding agent behavior requires methods that scale to thousands of trajectories and surface new patterns in long, often unfamiliar tasks where pre-built classifiers fall short. We propose to bring grounded theory i…

Disentangling Representations through Multi-task Learning

2024-07-15 · Pantelis Vafidis, Aman Bhargava, Antonio Rangel

Intelligent perception and interaction with the world hinges on internal representations that capture its underlying structure (''disentangled'' or ''abstract'' representations). Disentangled representations serve as wor…

Decision Making Multi-Task Learning Zero-shot Generalization

Can We Really Learn One Representation to Optimize All Rewards?

2026-02-11 · Chongyi Zheng, Royina Karegoudra Jayanth, Benjamin Eysenbach arxiv

As unsupervised pretraining becomes increasingly ubiquitous in reinforcement learning, a more thorough theoretical understanding of these methods becomes of equal importance to their empirical success. We focus on the se…

Representation Learning Reinforcement Learning Continuous Control