ES-MAML: Simple Hessian-Free Meta Learning
We introduce ES-MAML, a new framework for solving the model agnostic meta learning (MAML) problem based on Evolution Strategies (ES). Existing algorithms for MAML are based on policy gradients, and incur significant difficulties when attempting to estimate second derivatives using backpropagation on stochastic policies. We show how ES can be applied to MAML to obtain an algorithm which avoids the problem of estimating second derivatives, and is also conceptually simple and easy to implement. Moreover, ES-MAML can handle new types of nonsmooth adaptation operators, and other techniques for improving performance and estimation of ES methods become applicable. We show empirically that ES-MAML is competitive with existing methods and often yields better adaptation with fewer queries.
Code (1)
Tasks
Meta-LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms
We study the convergence of a class of gradient-based Model-Agnostic Meta-Learning (MAML) methods and characterize their overall complexity as well as their best achievable accuracy in terms of gradient norm for nonconve…
Meta-LearningEfficient Meta Learning via Minibatch Proximal Update
We address the problem of meta-learning which learns a prior over hypothesis from a sample of meta-training tasks for fast adaptation on meta-testing tasks. A particularly simple yet successful paradigm for this research…
Few-Shot LearningMeta-LearningMeta-Learning for Federated Face Recognition in Imbalanced Data Regimes
The growing privacy concerns surrounding face image data demand new techniques that can guarantee user privacy. One such face recognition technique that claims to achieve better user privacy is Federated Face Recognition…
Face RecognitionFairnessFederated LearningMeta-LearningFORML: A Riemannian Hessian-free Method for Meta-learning on Stiefel Manifolds
Meta-learning problem is usually formulated as a bi-level optimization in which the task-specific and the meta-parameters are updated in the inner and outer loops of optimization, respectively. However, performing the op…
Few-Shot LearningMeta-LearningConvergence of First-Order Algorithms for Meta-Learning with Moreau Envelopes
In this work, we consider the problem of minimizing the sum of Moreau envelopes of given functions, which has previously appeared in the context of meta-learning and personalized federated learning. In contrast to the ex…
Federated LearningMeta-LearningPersonalized Federated Learning