Regularized Context Gates on Transformer for Machine Translation
Context gates are effective to control the contributions from the source and target contexts in the recurrent neural network (RNN) based neural machine translation (NMT). However, it is challenging to extend them into the advanced Transformer architecture, which is more complicated than RNN. This paper first provides a method to identify source and target contexts and then introduce a gate mechanism to control the source and target contributions in Transformer. In addition, to further reduce the bias problem in the gate mechanism, this paper proposes a regularization method to guide the learning of the gates with supervision automatically generated using pointwise mutual information. Extensive experiments on 4 translation datasets demonstrate that the proposed model obtains an averaged gain of 1.0 BLEU score over a strong Transformer baseline.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationNMTTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Exploiting Sentential Context for Neural Machine Translation
In this work, we present novel approaches to exploit sentential context for neural machine translation (NMT). Specifically, we first show that a shallow sentential context extracted from the top encoder layer only, can i…
Machine TranslationNMTTranslationEncoding Sentence Position in Context-Aware Neural Machine Translation with Concatenation
Context-aware translation can be achieved by processing a concatenation of consecutive sentences with the standard Transformer architecture. This paper investigates the intuitive idea of providing the model with explicit…
Machine TranslationPositionSentenceTranslationHW-TSC's Submission to the CCMT 2024 Machine Translation Tasks
This paper presents the submission of Huawei Translation Services Center (HW-TSC) to machine translation tasks of the 20th China Conference on Machine Translation (CCMT 2024). We participate in the bilingual machine tran…
Automatic Post-EditingEnsemble LearningLanguage ModelingLanguage Modelling+4Context Gates for Neural Machine Translation
In neural machine translation (NMT), generation of a target word depends on both source and target contexts. We find that source contexts have a direct impact on the adequacy of a translation while target contexts affect…
Machine TranslationNMTTranslationSingle Model Ensemble for Subword Regularized Models in Low-Resource Machine Translation
Subword regularizations use multiple subword segmentations during training to improve the robustness of neural machine translation models. In previous subword regularizations, we use multiple segmentations in the trainin…
Machine TranslationSegmentationTranslation