Non-autoregressive Machine Translation with Probabilistic Context-free Grammar
Non-autoregressive Transformer(NAT) significantly accelerates the inference of neural machine translation. However, conventional NAT models suffer from limited expression power and performance degradation compared to autoregressive (AT) models due to the assumption of conditional independence among target tokens. To address these limitations, we propose a novel approach called PCFG-NAT, which leverages a specially designed Probabilistic Context-Free Grammar (PCFG) to enhance the ability of NAT models to capture complex dependencies among output tokens. Experimental results on major machine translation benchmarks demonstrate that PCFG-NAT further narrows the gap in translation quality between NAT and AT models. Moreover, PCFG-NAT facilitates a deeper understanding of the generated sentences, addressing the lack of satisfactory explainability in neural machine translation.Code is publicly available at https://github.com/ictnlp/PCFG-NAT.
Code (1)
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
Streaming Translation and Transcription Through Speech-to-Text Causal Alignment
Simultaneous machine translation (SiMT) has traditionally relied on offline machine translation models coupled with human-engineered heuristics or learned policies. We propose Hikari, a policy-free, fully end-to-end mode…
Speech-to-Text TranslationMachine TranslationImitation Learning for Non-Autoregressive Neural Machine Translation
Non-autoregressive translation models (NAT) have achieved impressive inference speedup. A potential issue of the existing NAT algorithms, however, is that the decoding is conducted in parallel, without directly consideri…
Imitation LearningMachine TranslationSentenceTranslationImproving Autoregressive NMT with Non-Autoregressive Model
Autoregressive neural machine translation (NMT) models are often used to teach non-autoregressive models via knowledge distillation. However, there are few studies on improving the quality of autoregressive translation (…
Decoderde-enKnowledge DistillationMachine Translation+3Revisiting the Markov Property for Machine Translation
In this paper, we re-examine the Markov property in the context of neural machine translation. We design a Markov Autoregressive Transformer~(MAT) and undertake a comprehensive assessment of its performance across four W…
Machine TranslationTranslationZero-Shot Translation using Diffusion Models
In this work, we show a novel method for neural machine translation (NMT), using a denoising diffusion probabilistic model (DDPM), adjusted for textual data, following recent advances in the field. We show that it's poss…
DenoisingMachine TranslationNMTSentence+2