Generalising via Meta-Examples for Continual Learning in the Wild
Future deep learning systems call for techniques that can deal with the evolving nature of temporal data and scarcity of annotations when new problems occur. As a step towards this goal, we present FUSION (Few-shot UnSupervIsed cONtinual learning), a learning strategy that enables a neural network to learn quickly and continually on streams of unlabelled data and unbalanced tasks. The objective is to maximise the knowledge extracted from the unlabelled data stream (unsupervised), favor the forward transfer of previously learnt tasks and features (continual) and exploit as much as possible the supervised information when available (few-shot). The core of FUSION is MEML - Meta-Example Meta-Learning - that consolidates a meta-representation through the use of a self-attention mechanism during a single inner loop in the meta-optimisation stage. To further enhance the capability of MEML to generalise from few data, we extend it by creating various augmented surrogate tasks and by optimising over the hardest. An extensive experimental evaluation on public computer vision benchmarks shows that FUSION outperforms existing state-of-the-art solutions both in the few-shot and continual learning experimental settings.
Code (1)
Tasks
Continual LearningFew-Shot LearningMeta-LearningSimilar Papers 제목 키워드 기반
Learning to Learn for Few-shot Continual Active Learning
Continual learning strives to ensure stability in solving previously seen tasks while demonstrating plasticity in a novel domain. Recent advances in continual learning are mostly confined to a supervised learning setting…
Active LearningContinual LearningMeta-Learningtext-classification+1Efficient Meta-Learning for Continual Learning with Taylor Expansion Approximation
Continual learning aims to alleviate catastrophic forgetting when handling consecutive tasks under non-stationary distributions. Gradient-based meta-learning algorithms have shown the capability to implicitly solve the t…
Computational EfficiencyContinual LearningMeta-LearningARCADe: A Rapid Continual Anomaly Detector
Although continual learning and anomaly detection have separately been well-studied in previous works, their intersection remains rather unexplored. The present work addresses a learning scenario where a model has to inc…
Anomaly Detectioncontinual anomaly detectionContinual LearningMeta-Learning+1Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time
Distribution shift occurs when the test distribution differs from the training distribution, and it can considerably degrade performance of machine learning models deployed in the real world. Temporal shifts -- distribut…
Continual LearningDomain GeneralizationEnsemble LearningNews Classification+2Hypernetworks for Continual Semi-Supervised Learning
Learning from data sequentially arriving, possibly in a non i.i.d. way, with changing task distribution over time is called continual learning. Much of the work thus far in continual learning focuses on supervised learni…
Continual LearningGenerative Adversarial NetworkSplit-CIFAR-10