Meta-Teaching: Curriculum Generation for Lifelong Learning
Meta-learning will be crucial to creating lifelong, generalizable AI. In practice, however, it is hard to define the meta-training task distribution that is used to train meta-learners. If made too small, tasks are too similar for a model to meaningfully generalize. If made too large, generalization becomes incredibly difficult. We argue that both problems can be alleviated by introducing a teacher model that controls the sequence of tasks that a meta-learner is trained on. This teacher model is incentivized to start the student meta-learner on simple tasks then adaptively increase task difficulty in response to student progress. While this approach has been previously studied in curriculum generation, our main contribution is in extending it to meta-learning.
Code (0)
등록된 구현이 없습니다.
Tasks
Lifelong learningMeta-LearningSimilar Papers 제목 키워드 기반
How Do Humans Teach: On Curriculum Learning and Teaching Dimension
We study the empirical strategies that humans follow as they teach a target concept with a simple 1D threshold to a robot. Previous studies of computational teaching, particularly the teaching dimension model and the cu…
LiBOG: Lifelong Learning for Black-Box Optimizer Generation
Meta-Black-Box Optimization (MetaBBO) garners attention due to its success in automating the configuration and generation of black-box optimizers, significantly reducing the human effort required for optimizer design and…
Lifelong learningTeaching to Teach by Structured Dark Knowledge
To educate hyper deep learners, \emph{Curriculum Learnings} (CLs) require either human heuristic participation or self-deciding the difficulties of training instances. These coaching manners are blind to the coherent str…
DiversityCurriculum Graph Co-Teaching for Multi-Target Domain Adaptation
In this paper we address multi-target domain adaptation (MTDA), where given one labeled source dataset and multiple unlabeled target datasets that differ in data distributions, the task is to learn a robust predictor for…
Blended-target Domain AdaptationDomain AdaptationMulti-target Domain AdaptationA Survey on Recent Teacher-student Learning Studies
Knowledge distillation is a method of transferring the knowledge from a complex deep neural network (DNN) to a smaller and faster DNN, while preserving its accuracy. Recent variants of knowledge distillation include teac…
Knowledge DistillationSurvey