Self-Paced Learning for Neural Machine Translation
Recent studies have proven that the training of neural machine translation (NMT) can be facilitated by mimicking the learning process of humans. Nevertheless, achievements of such kind of curriculum learning rely on the quality of artificial schedule drawn up with the handcrafted features, e.g. sentence length or word rarity. We ameliorate this procedure with a more flexible manner by proposing self-paced learning, where NMT model is allowed to 1) automatically quantify the learning confidence over training examples; and 2) flexibly govern its learning via regulating the loss in each iteration step. Experimental results over multiple translation tasks demonstrate that the proposed model yields better performance than strong baselines and those models trained with human-designed curricula on both translation quality and convergence speed.
Code (1)
Tasks
Machine TranslationNMTSentenceTranslationSimilar Papers 제목 키워드 기반
Reward-Machine-Guided, Self-Paced Reinforcement Learning
Self-paced reinforcement learning (RL) aims to improve the data efficiency of learning by automatically creating sequences, namely curricula, of probability distributions over contexts. However, existing techniques for s…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Can the Variation of Model Weights be used as a Criterion for Self-Paced Multilingual NMT?
Many-to-one neural machine translation systems improve over one-to-one systems when training data is scarce. In this paper, we design and test a novel algorithm for selecting the language of minibatches when training suc…
Machine TranslationNMTTranslationSelf-paced ensemble learning for speech and audio classification
Combining multiple machine learning models into an ensemble is known to provide superior performance levels compared to the individual components forming the ensemble. This is because models can complement each other in …
Audio ClassificationClassificationEnsemble LearningGeneral Classification+1Balanced Self-Paced Learning for AUC Maximization
Learning to improve AUC performance is an important topic in machine learning. However, AUC maximization algorithms may decrease generalization performance due to the noisy data. Self-paced learning is an effective metho…
Training with Growing Sets: A Simple Alternative to Curriculum Learning and Self Paced Learning
Curriculum learning and Self paced learning are popular topics in the machine learning that suggest to put the training samples in order by considering their difficulty levels. Studies in these topics show that starting …