Meta-SGD: Learning to Learn Quickly for Few-Shot Learning

2017-07-31 · Zhenguo Li, Fengwei Zhou, Fei Chen, Hang Li

Few-shot learning is challenging for learning algorithms that learn each task in isolation and from scratch. In contrast, meta-learning learns from many related tasks a meta-learner that can learn a new task more accurately and faster with fewer examples, where the choice of meta-learners is crucial. In this paper, we develop Meta-SGD, an SGD-like, easily trainable meta-learner that can initialize and adapt any differentiable learner in just one step, on both supervised learning and reinforcement learning. Compared to the popular meta-learner LSTM, Meta-SGD is conceptually simpler, easier to implement, and can be learned more efficiently. Compared to the latest meta-learner MAML, Meta-SGD has a much higher capacity by learning to learn not just the learner initialization, but also the learner update direction and learning rate, all in a single meta-learning process. Meta-SGD shows highly competitive performance for few-shot learning on regression, classification, and reinforcement learning.

📄 PDF Abstract BibTeX arXiv:1707.09835

Code (9)

BBDrive/Meta-SGD-RL pytorch

ash3n/Meta-Gradients tf

ash3n/Meta-SGD tf

ash3n/Meta-SGD-TF tf

foolyc/Meta-SGD tf

learnables/learn2learn pytorch

llan-ml/tesp tf

myungsub/meta-interpolation pytorch

tobiasvanderwerff/MetaHTR pytorch

Tasks

Few-Shot Learning Meta-Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

MAML 설명 없음

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

Meta-Graph: Few Shot Link Prediction via Meta Learning

2019-12-20 · Avishek Joey Bose, Ankit Jain, Piero Molino, William L. Hamilton

We consider the task of few shot link prediction on graphs. The goal is to learn from a distribution over graphs so that a model is able to quickly infer missing edges in a new graph after a small amount of training. We …

Graph Neural Network Link Prediction Meta-Learning Prediction

Learning to Learn to Disambiguate: Meta-Learning for Few-Shot Word Sense Disambiguation

2020-04-29 · Findings of the Association for Computational Linguistics 2020 · Nithin Holla, Pushkar Mishra, Helen Yannakoudakis, Ekaterina Shutova

The success of deep learning methods hinges on the availability of large training datasets annotated for the task of interest. In contrast to human intelligence, these methods lack versatility and struggle to learn and a…

Few-Shot Learning Meta-Learning Word Sense Disambiguation

A Theoretical Analysis of the Number of Shots in Few-Shot Learning

2019-09-25 · ICLR 2020 1 · Tianshi Cao, Marc Law, Sanja Fidler

Few-shot classification is the task of predicting the category of an example from a set of few labeled examples. The number of labeled examples per category is called the number of shots (or shot number). Recent works ta…

Classification Few-Shot Learning General Classification Meta-Learning

MetaSLRCL: A Self-Adaptive Learning Rate and Curriculum Learning Based Framework for Few-Shot Text Classification

2022-10-01 · COLING 2022 10 · Kailin Zhao, Xiaolong Jin, Saiping Guan, Jiafeng Guo 외

Due to the lack of labeled data in many realistic scenarios, a number of few-shot learning methods for text classification have been proposed, among which the meta learning based ones have recently attracted much attenti…

Few-Shot Learning Few-Shot Text Classification Meta-Learning text-classification+1

A Self-Adaptive Learning Rate and Curriculum Learning Based Framework for Few-Shot Text Classification

2021-11-16 · ACL ARR November 2021 11 · Anonymous