Scheduled Multi-Task Learning: From Syntax to Translation
Neural encoder-decoder models of machine translation have achieved impressive results, while learning linguistic knowledge of both the source and target languages in an implicit end-to-end manner. We propose a framework in which our model begins learning syntax and translation interleaved, gradually putting more focus on translation. Using this approach, we achieve considerable improvements in terms of BLEU score on relatively large parallel corpus (WMT14 English to German) and a low-resource (WIT German to English) setup.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderMachine TranslationMulti-Task LearningTranslationSimilar Papers 제목 키워드 기반
Adaptively Scheduled Multitask Learning: The Case of Low-Resource Neural Machine Translation
Neural Machine Translation (NMT), a data-hungry technology, suffers from the lack of bilingual data in low-resource scenarios. Multitask learning (MTL) can alleviate this issue by injecting inductive biases into NMT, usi…
Low Resource Neural Machine TranslationLow-Resource Neural Machine TranslationMachine TranslationNMT+1Scheduled Multi-task Learning for Neural Chat Translation
Neural Chat Translation (NCT) aims to translate conversational text into different languages. Existing methods mainly focus on modeling the bilingual dialogue characteristics (e.g., coherence) to improve chat translation…
Multi-Task LearningTranslationScheduled Sampling Based on Decoding Steps for Neural Machine Translation
Scheduled sampling is widely used to mitigate the exposure bias problem for neural machine translation. Its core motivation is to simulate the inference scene during training by replacing ground-truth tokens with predict…
Machine TranslationText SummarizationTranslationMixed Cross Entropy Loss for Neural Machine Translation
In neural machine translation, cross entropy (CE) is the standard loss function in two training methods of auto-regressive models, i.e., teacher forcing and scheduled sampling. In this paper, we propose mixed cross entro…
Machine TranslationTranslationPredicting Target Language CCG Supertags Improves Neural Machine Translation
Neural machine translation (NMT) models are able to partially learn syntactic information from sequential lexical information. Still, some complex syntactic phenomena such as prepositional phrase attachment are poorly mo…
DecoderMachine TranslationNMTPrepositional Phrase Attachment+1