Dynamic Data Selection for Curriculum Learning via Ability Estimation
Curriculum learning methods typically rely on heuristics to estimate the difficulty of training examples or the ability of the model. In this work, we propose replacing difficulty heuristics with learned difficulty parameters. We also propose Dynamic Data selection for Curriculum Learning via Ability Estimation (DDaCLAE), a strategy that probes model ability at each training epoch to select the best training examples at that point. We show that models using learned difficulty and/or ability outperform heuristic-based curriculum learning models on the GLUE classification tasks.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Curriculum Manager for Source Selection in Multi-Source Domain Adaptation
The performance of Multi-Source Unsupervised Domain Adaptation depends significantly on the effectiveness of transfer from labeled source domain samples. In this paper, we proposed an adversarial agent that learns a dyna…
Domain AdaptationMulti-Source Unsupervised Domain AdaptationUnsupervised Domain AdaptationDynamically Composing Domain-Data Selection with Clean-Data Selection by "Co-Curricular Learning" for Neural Machine Translation
Noise and domain are important aspects of data quality for neural machine translation. Existing research focus separately on domain-data selection, clean-data selection, or their static combination, leaving the dynamic i…
Machine TranslationTransfer LearningTranslationDynamically Composing Domain-Data Selection with Clean-Data Selection by ``Co-Curricular Learning'' for Neural Machine Translation
Noise and domain are important aspects of data quality for neural machine translation. Existing research focus separately on domain-data selection, clean-data selection, or their static combination, leaving the dynamic i…
Machine TranslationTransfer LearningTranslationA Psychology-based Unified Dynamic Framework for Curriculum Learning
Directly learning from examples of random difficulty levels is often challenging for both humans and machine learning models. A more effective strategy involves exposing learners to examples in a progressive order, from …
Refining Context-Entangled Content Segmentation via Curriculum Selection and Anti-Curriculum Promotion
Biological learning proceeds from easy to difficult tasks, gradually reinforcing perception and robustness. Inspired by this principle, we address Context-Entangled Content Segmentation (CECS), a challenging setting wher…
Object Detection