Neural Machine Translation via Binary Code Prediction
In this paper, we propose a new method for calculating the output layer in neural machine translation systems. The method is based on predicting a binary code for each word and can reduce computation time/memory requirements of the output layer to be logarithmic in vocabulary size in the best case. In addition, we also introduce two advanced approaches to improve the robustness of the proposed model: using error-correcting codes and combining softmax and binary codes. Experiments on two English-Japanese bidirectional translation tasks show proposed models achieve BLEU scores that approach the softmax, while reducing memory usage to the order of less than 1/10 and improving decoding speed on CPUs by x5 to x10.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationPredictionTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Beyond Binary Gender: Evaluating Gender-Inclusive Machine Translation with Ambiguous Attitude Words
Gender bias has been a focal point in the study of bias in machine translation and language models. Existing machine translation gender bias evaluations are primarily focused on male and female genders, limiting the scop…
Machine TranslationTranslationNeural Machine Translation for Code Generation
Neural machine translation (NMT) methods developed for natural language processing have been shown to be highly successful in automating translation from one natural language to another. Recently, these NMT methods have …
Code CompletionCode GenerationCode TranslationMachine Translation+2QEMind: Alibaba's Submission to the WMT21 Quality Estimation Shared Task
Quality Estimation, as a crucial step of quality control for machine translation, has been explored for years. The goal is to investigate automatic methods for estimating the quality of machine translation results withou…
Machine TranslationSentenceTranslationQEMind: Alibaba’s Submission to the WMT21 Quality Estimation Shared Task
Quality Estimation, as a crucial step of quality control for machine translation, has been explored for years. The goal is to to investigate automatic methods for estimating the quality of machine translation results wit…
Machine TranslationSentenceTranslationLearning Optimal Policy for Simultaneous Machine Translation via Binary Search
Simultaneous machine translation (SiMT) starts to output translation while reading the source sentence and needs a precise policy to decide when to output the generated translation. Therefore, the policy determines the n…
Machine TranslationSentenceTranslation