Robust Meta-learning with Noise via Eigen-Reptile
Recent years have seen a surge of interest in meta-learning techniques for tackling the few-shot learning (FSL) problem. However, the meta-learner's initial model is prone to meta-overfit, as there are only a few available samples with sampling noise. Besides, when handling the data sampled with label noise for FSL, meta-learner could be extremely sensitive to label noise. To address these two challenges that FSL with sampling and label noise. In particular, we first cast the meta-overfitting problem (overfitting on sampling and label noise) as a gradient noise problem since few available samples cause meta-learner to overfit on existing examples (clean or corrupted) of an individual task at every gradient step. We present Eigen-Reptile (ER) that updates the meta-parameters with the main direction of historical task-specific parameters to alleviate gradient noise. Specifically, the main direction is computed by a special mechanism for the parameter's large size. Furthermore, to obtain a more accurate main direction for Eigen-Reptile in the presence of label noise, we propose Introspective Self-paced Learning (ISPL) that constructs a plurality of prior models to determine which sample should be abandoned. We have proved the effectiveness of Eigen-Reptile and ISPL, respectively, theoretically and experimentally. Moreover, our experiments on different tasks demonstrate that the proposed methods outperform or achieve highly competitive performance compared with the state-of-the-art methods with or without noisy labels.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot LearningMeta-LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptile
Recent years have seen a surge of interest in meta-learning techniques for tackling the few-shot learning (FSL) problem. However, the meta-learner is prone to overfitting since there are only a few available samples, whi…
Few-Shot LearningMeta-LearningEEG-Reptile: An Automatized Reptile-Based Meta-Learning Library for BCIs
Meta-learning, i.e., "learning to learn", is a promising approach to enable efficient BCI classifier training with limited amounts of data. It can effectively use collections of in some way similar classification tasks, …
EEGFew-Shot LearningMeta-LearningTransfer LearningUnderstanding Transfer Learning and Gradient-Based Meta-Learning Techniques
Deep neural networks can yield good performance on various tasks but often require large amounts of data to train them. Meta-learning received considerable attention as one approach to improve the generalization of these…
DiversityMeta-LearningTransfer LearningMeta-Learning for Few-Shot Named Entity Recognition
Meta-learning has recently been proposed to learn models and algorithms that can generalize from a handful of examples. However, applications to structured prediction and textual tasks pose challenges for meta-learning a…
Language ModelingLanguage ModellingMeta-Learningnamed-entity-recognition+4RAMario: Experimental Approach to Reptile Algorithm -- Reinforcement Learning for Mario
This research paper presents an experimental approach to using the Reptile algorithm for reinforcement learning to train a neural network to play Super Mario Bros. We implement the Reptile algorithm using the Super Mario…
Few-Shot LearningMeta-Learningreinforcement-learningReinforcement Learning