When does MAML Work the Best? An Empirical Study on Model-Agnostic Meta-Learning in NLP Applications
Model-Agnostic Meta-Learning (MAML), a model-agnostic meta-learning method, is successfully employed in NLP applications including few-shot text classification and multi-domain low-resource language generation. Many impacting factors, including data quantity, similarity among tasks, and the balance between general language model and task-specific adaptation, can affect the performance of MAML in NLP, but few works have thoroughly studied them. In this paper, we conduct an empirical study to investigate these impacting factors and conclude when MAML works the best based on the experimental results.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot Text ClassificationLanguage ModelingLanguage ModellingMeta-Learningtext-classificationText ClassificationText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning
Model-agnostic meta learning (MAML) is currently one of the dominating approaches for few-shot meta-learning. Albeit its effectiveness, the optimization of MAML can be challenging due to the innate bilevel problem struct…
Meta-LearningmodelWhen Does MAML Objective Have Benign Landscape?
The paper studies the complexity of the optimization problem behind the Model-Agnostic Meta-Learning (MAML) algorithm. The goal of the study is to determine the global convergence of MAML on sequential decision-making ta…
Decision MakingMeta-LearningSequential Decision MakingIs Pre-training Truly Better Than Meta-Learning?
In the context of few-shot learning, it is currently believed that a fixed pre-trained (PT) model, along with fine-tuning the final layer during evaluation, outperforms standard meta-learning algorithms. We re-evaluate t…
DiversityFew-Shot LearningMeta-LearningIs Bayesian Model-Agnostic Meta Learning Better than Model-Agnostic Meta Learning, Provably?
Meta learning aims at learning a model that can quickly adapt to unseen tasks. Widely used meta learning methods include model agnostic meta learning (MAML), implicit MAML, Bayesian MAML. Thanks to its ability of modelin…
Meta-LearningmodelDoes MAML Only Work via Feature Re-use? A Data Centric Perspective
Recent work has suggested that a good embedding is all we need to solve many few-shot learning benchmarks. Furthermore, other work has strongly suggested that Model Agnostic Meta-Learning (MAML) also works via this same …
Few-Shot LearningMeta-Learning