On Modulating the Gradient for Meta-Learning
Inspired by optimization techniques, we propose a novel meta-learning algorithm with gradient modulation to encourage fast-adaptation of neural networks in the absence of abundant data. Our method, termed ModGrad, is designed to circumvent the noisy nature of the gradients which is prevalent in low-data regimes. Furthermore and having the scalability concern in mind, we formulate ModGrad via low-rank approximations, which in turn enables us to employ ModGrad to adapt hefty neural networks. We thoroughly assess and contrast ModGrad against a large family of meta-learning techniques and observe that the proposed algorithm outperforms baselines comfortably while enjoying faster convergence.
Code (1)
Tasks
Meta-LearningSimilar Papers 제목 키워드 기반
Meta Gradient Boosting Neural Networks
Meta-optimization is an effective approach that learns a shared set of parameters across tasks for parameter initialization in meta-learning. A key challenge for meta-optimization based approaches is to determine whether…
Meta-LearningregressionToward Multimodal Model-Agnostic Meta-Learning
Gradient-based meta-learners such as MAML are able to learn a meta-prior from similar tasks to adapt to novel tasks from the same distribution with few gradient updates. One important limitation of such frameworks is tha…
DiversityFew-Shot Image Classificationimage-classificationImage Classification+3Model-Agnostic Meta-Learning for Multimodal Task Distributions
Gradient-based meta-learners such as MAML (Finn et al., 2017) are able to learn a meta-prior from similar tasks to adapt to novel tasks from the same distribution with few gradient updates. One important limitation of su…
DiversityFew-Shot Image Classificationimage-classificationImage Classification+2Modulating transfer between tasks in gradient-based meta-learning
Learning-to-learn or meta-learning leverages data-driven inductive bias to increase the efficiency of learning on a novel task. This approach encounters difficulty when transfer is not mutually beneficial, for instance, …
DiversityInductive BiasMeta-LearningMultimodal Model-Agnostic Meta-Learning via Task-Aware Modulation
Model-agnostic meta-learners aim to acquire meta-learned parameters from similar tasks to adapt to novel tasks from the same distribution with few gradient updates. With the flexibility in the choice of models, those fra…
Few-Shot Image ClassificationFew-Shot LearningGeneral Classificationimage-classification+5