Factorized Transformer for Multi-Domain Neural Machine Translation
Multi-Domain Neural Machine Translation (NMT) aims at building a single system that performs well on a range of target domains. However, along with the extreme diversity of cross-domain wording and phrasing style, the imperfections of training data distribution and the inherent defects of the current sequential learning process all contribute to making the task of multi-domain NMT very challenging. To mitigate these problems, we propose the Factorized Transformer, which consists of an in-depth factorization of the parameters of an NMT model, namely Transformer in this paper, into two categories: domain-shared ones that encode common cross-domain knowledge and domain-specific ones that are private for each constituent domain. We experiment with various designs of our model and conduct extensive validations on English to French open multi-domain dataset. Our approach achieves state-of-the-art performance and opens up new perspectives for multi-domain and open-domain applications.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityMachine TranslationNMTTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
The AFRL IWSLT 2020 Systems: Work-From-Home Edition
This report summarizes the Air Force Research Laboratory (AFRL) submission to the offline spoken language translation (SLT) task as part of the IWSLT 2020 evaluation campaign. As in previous years, we chose to adopt the …
Action DetectionActivity DetectionAutomatic Speech RecognitionAutomatic Speech Recognition (ASR)+9CaFA: Global Weather Forecasting with Factorized Attention on Sphere
Accurate weather forecasting is crucial in various sectors, impacting decision-making processes and societal events. Data-driven approaches based on machine learning models have recently emerged as a promising alternativ…
PredictionWeather ForecastingScalable Transformer for PDE Surrogate Modeling
Transformer has shown state-of-the-art performance on various applications and has recently emerged as a promising tool for surrogate modeling of partial differential equations (PDEs). Despite the introduction of linear-…
PDE Surrogate ModelingFactorized Multimodal Transformer for Multimodal Sequential Learning
The complex world around us is inherently multimodal and sequential (continuous). Information is scattered across different modalities and requires multiple continuous sensors to be captured. As machine learning leaps to…
NVIDIA NeMo Neural Machine Translation Systems for English-German and English-Russian News and Biomedical Tasks at WMT21
This paper provides an overview of NVIDIA NeMo's neural machine translation systems for the constrained data track of the WMT21 News and Biomedical Shared Translation Tasks. Our news task submissions for English-German (…
Data AugmentationKnowledge DistillationMachine TranslationRe-Ranking+1