Can the Variation of Model Weights be used as a Criterion for Self-Paced Multilingual NMT?
Many-to-one neural machine translation systems improve over one-to-one systems when training data is scarce. In this paper, we design and test a novel algorithm for selecting the language of minibatches when training such systems. The algorithm changes the language of the minibatch when the weights of the model do not evolve significantly, as measured by the smoothed KL divergence between all layers of the Transformer network. This algorithm outperforms the use of alternating monolingual batches, but not the use of shuffled batches, in terms of translation quality (measured with BLEU and COMET) and convergence speed.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationNMTTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Distributed Self-Paced Learning in Alternating Direction Method of Multipliers
Self-paced learning (SPL) mimics the cognitive process of humans, who generally learn from easy samples to hard ones. One key issue in SPL is the training process required for each instance weight depends on the other sa…
Deep Self-Paced Learning for Person Re-Identification
Person re-identification (Re-ID) usually suffers from noisy samples with background clutter and mutual occlusion, which makes it extremely difficult to distinguish different individuals across the disjoint camera views. …
Person Re-IdentificationTripletRobust Self-Paced Hashing for Cross-Modal Retrieval with Noisy Labels
Cross-modal hashing (CMH) has appeared as a popular technique for cross-modal retrieval due to its low storage cost and high computational efficiency in large-scale data. Most existing methods implicitly assume that mult…
Computational EfficiencyCross-Modal RetrievalRetrievalSpatially Self-Paced Convolutional Networks for Change Detection in Heterogeneous Images
Change detection in heterogeneous remote sensing images is a challenging problem because it is hard to make a direct comparison in the original observation spaces, and most methods rely on a set of manually labeled sampl…
Change DetectionSimple and Robust Loss Design for Multi-Label Learning with Missing Labels
Multi-label learning in the presence of missing labels (MLML) is a challenging problem. Existing methods mainly focus on the design of network structures or training schemes, which increase the complexity of implementati…
image-classificationImage ClassificationMissing LabelsMulti-Label Image Classification+1