On the Subspace Structure of Gradient-Based Meta-Learning
In this work we provide an analysis of the distribution of the post-adaptation parameters of Gradient-Based Meta-Learning (GBML) methods. Previous work has noticed how, for the case of image-classification, this adaptation only takes place on the last layers of the network. We propose the more general notion that parameters are updated over a low-dimensional \emph{subspace} of the same dimensionality as the task-space and show that this holds for regression as well. Furthermore, the induced subspace structure provides a method to estimate the intrinsic dimension of the space of tasks of common few-shot learning datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot Learningimage-classificationImage ClassificationMeta-LearningregressionSimilar Papers 제목 키워드 기반
Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspace
Gradient-based meta-learning methods leverage gradient descent to learn the commonalities among various tasks. While previous such methods have been successful in meta-learning tasks, they resort to simple gradient desce…
Few-Shot Image ClassificationMeta-LearningLarge-Scale Riemannian Meta-Optimization via Subspace Adaptation
Riemannian meta-optimization provides a promising approach to solving non-linear constrained optimization problems, which trains neural networks as optimizers to perform optimization on Riemannian manifolds. However, exi…
Multi-Subspace Structured Meta-Learning
Meta-learning aims to extract meta-knowledge from historical tasks to accelerate learning on new tasks. A critical challenge in meta-learning is to handle task heterogeneity, i.e., tasks lie in different distributions. U…
Meta-LearningSubspace Adaptation Prior for Few-Shot Learning
Gradient-based meta-learning techniques aim to distill useful prior knowledge from a set of training tasks such that new tasks can be learned more efficiently with gradient descent. While these methods have achieved succ…
Few-Shot Image ClassificationFew-Shot Learningimage-classificationImage Classification+1Online Supervised Subspace Tracking
We present a framework for supervised subspace tracking, when there are two time series $x_t$ and $y_t$, one being the high-dimensional predictors and the other being the response variables and the subspace tracking need…
Dimensionality ReductionregressionTime SeriesTime Series Analysis