More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithms
We introduce a new framework for studying meta-learning methods using PAC-Bayesian theory. Its main advantage over previous work is that it allows for more flexibility in how the transfer of knowledge between tasks is realized. For previous approaches, this could only happen indirectly, by means of learning prior distributions over models. In contrast, the new generalization bounds that we prove express the process of meta-learning much more directly as learning the learning algorithm that should be used for future tasks. The flexibility of our framework makes it suitable to analyze a wide range of meta-learning mechanisms and even design new mechanisms. Other than our theoretical contributions we also show empirically that our framework improves the prediction quality in practical meta-learning mechanisms.
Code (1)
Tasks
Generalization BoundsMeta-LearningSimilar Papers 제목 키워드 기반
Meta-Learning for Simple Regret Minimization
We develop a meta-learning framework for simple regret minimization in bandits. In this framework, a learning agent interacts with a sequence of bandit tasks, which are sampled i.i.d.\ from an unknown prior distribution,…
Meta-LearningMulti-objective Bayesian Optimization with Human-in-the-Loop for Flexible Neuromorphic Electronics Fabrication
Neuromorphic computing hardware enables edge computing and can be implemented in flexible electronics for novel applications. Metal oxide materials are promising candidates for fabricating flexible neuromorphic electroni…
Amortising Inference and Meta-Learning Priors in Neural Networks
One of the core facets of Bayesianism is in the updating of prior beliefs in light of new evidence$\text{ -- }$so how can we maintain a Bayesian approach if we have no prior beliefs in the first place? This is one of the…
Meta-Learned Models of Cognition
Meta-learning is a framework for learning learning algorithms through repeated interactions with an environment as opposed to designing them by hand. In recent years, this framework has established itself as a promising …
Bayesian InferenceMeta-LearningBridging the Gap Between Practice and PAC-Bayes Theory in Few-Shot Meta-Learning
Despite recent advances in its theoretical understanding, there still remains a significant gap in the ability of existing PAC-Bayesian theories on meta-learning to explain performance improvements in the few-shot learni…
Few-Shot LearningMeta-Learning