ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learning
Optimization-based meta-learning typically assumes tasks are sampled from a single distribution - an assumption oversimplifies and limits the diversity of tasks that meta-learning can model. Handling tasks from multiple different distributions is challenging for meta-learning due to a so-called task ambiguity issue. This paper proposes a novel method, ST-MAML, that empowers model-agnostic meta-learning (MAML) to learn from multiple task distributions. ST-MAML encodes tasks using a stochastic neural network module, that summarizes every task with a stochastic representation. The proposed Stochastic Task (ST) strategy allows a meta-model to get tailored for the current task and enables us to learn a distribution of solutions for an ambiguous task. ST-MAML also propagates the task representation to revise the encoding of input variables. Empirically, we demonstrate that ST-MAML matches or outperforms the state-of-the-art on two few-shot image classification tasks, one curve regression benchmark, one image completion problem, and a real-world temperature prediction application. To the best of authors' knowledge, this is the first time optimization-based meta-learning method being applied on a large-scale real-world task.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityFew-Shot Image Classificationimage-classificationImage ClassificationMeta-LearningSimilar Papers 제목 키워드 기반
HetMAML: Task-Heterogeneous Model-Agnostic Meta-Learning for Few-Shot Learning Across Modalities
Existing gradient-based meta-learning approaches to few-shot learning assume that all tasks have the same input feature space. However, in the real world scenarios, there are many cases that the input structures of tasks…
Few-Shot LearningMeta-LearningVocal Bursts Type PredictionDirected-MAML: Meta Reinforcement Learning Algorithm with Task-directed Approximation
Model-Agnostic Meta-Learning (MAML) is a versatile meta-learning framework applicable to both supervised learning and reinforcement learning (RL). However, applying MAML to meta-reinforcement learning (meta-RL) presents …
Computational EfficiencyReinforcement LearningOn the Convergence Theory of Debiased Model-Agnostic Meta-Reinforcement Learning
We consider Model-Agnostic Meta-Learning (MAML) methods for Reinforcement Learning (RL) problems, where the goal is to find a policy using data from several tasks represented by Markov Decision Processes (MDPs) that can …
Meta-LearningMeta Reinforcement LearningPolicy Gradient Methodsreinforcement-learning+2Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning
In recent years, model-agnostic meta-learning (MAML) has become a popular research area. However, the stochastic optimization of MAML is still underdeveloped. Existing MAML algorithms rely on the ``episode'' idea by samp…
Continual LearningFederated LearningMeta-LearningPersonalized Federated Learning+1Energy-Efficient and Federated Meta-Learning via Projected Stochastic Gradient Ascent
In this paper, we propose an energy-efficient federated meta-learning framework. The objective is to enable learning a meta-model that can be fine-tuned to a new task with a few number of samples in a distributed setting…
image-classificationImage ClassificationMeta-Learning