Towards Decoding as Continuous Optimization in Neural Machine Translation
We propose a novel decoding approach for neural machine translation (NMT) based on continuous optimisation. We convert decoding - basically a discrete optimization problem - into a continuous optimization problem. The resulting constrained continuous optimisation problem is then tackled using gradient-based methods. Our powerful decoding framework enables decoding intractable models such as the intersection of left-to-right and right-to-left (bidirectional) as well as source-to-target and target-to-source (bilingual) NMT models. Our empirical results show that our decoding framework is effective, and leads to substantial improvements in translations generated from the intersected models where the typical greedy or beam search is not feasible. We also compare our framework against reranking, and analyse its advantages and disadvantages.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationNMTRerankingTranslationSimilar Papers 제목 키워드 기반
Towards Decoding as Continuous Optimisation in Neural Machine Translation
We propose a novel decoding approach for neural machine translation (NMT) based on continuous optimisation. We reformulate decoding, a discrete optimization problem, into a continuous problem, such that optimization can …
Machine TranslationNMTRerankingTranslationRelay Decoding: Concatenating Large Language Models for Machine Translation
Leveraging large language models for machine translation has demonstrated promising results. However, it does require the large language models to possess the capability of handling both the source and target languages i…
Machine TranslationTranslationDirect Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding
Minimum Bayes Risk (MBR) decoding can significantly improve translation performance of Multilingual Large Language Models (MLLMs). However, MBR decoding is computationally expensive. We show how the recently developed Re…
Machine TranslationNMTreinforcement-learningReinforcement Learning+2Speed-Constrained Tuning for Statistical Machine Translation Using Bayesian Optimization
We address the problem of automatically finding the parameters of a statistical machine translation system that maximize BLEU scores while ensuring that decoding speed exceeds a minimum value. We propose the use of Bayes…
Bayesian OptimizationMachine TranslationTranslationControlled Text Generation as Continuous Optimization with Multiple Constraints
As large-scale language model pretraining pushes the state-of-the-art in text generation, recent work has turned to controlling attributes of the text such models generate. While modifying the pretrained models via fine-…
Language ModelingLanguage ModellingMachine TranslationSentence+3