Lattice-Based Recurrent Neural Network Encoders for Neural Machine Translation
Neural machine translation (NMT) heavily relies on word-level modelling to learn semantic representations of input sentences. However, for languages without natural word delimiters (e.g., Chinese) where input sentences have to be tokenized first, conventional NMT is confronted with two issues: 1) it is difficult to find an optimal tokenization granularity for source sentence modelling, and 2) errors in 1-best tokenizations may propagate to the encoder of NMT. To handle these issues, we propose word-lattice based Recurrent Neural Network (RNN) encoders for NMT, which generalize the standard RNN to word lattice topology. The proposed encoders take as input a word lattice that compactly encodes multiple tokenizations, and learn to generate new hidden states from arbitrarily many inputs and hidden states in preceding time steps. As such, the word-lattice based encoders not only alleviate the negative impact of tokenization errors but also are more expressive and flexible to embed input sentences. Experiment results on Chinese-English translation demonstrate the superiorities of the proposed encoders over the conventional encoder.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationNMTSentenceTranslationSimilar Papers 제목 키워드 기반
Lattice-Based Transformer Encoder for Neural Machine Translation
Neural machine translation (NMT) takes deterministic sequences for source representations. However, either word-level or subword-level segmentations have multiple choices to split a source sequence with different word se…
DiversityMachine TranslationNMTTranslationNeural Lattice Search for Domain Adaptation in Machine Translation
Domain adaptation is a major challenge for neural machine translation (NMT). Given unknown words or new domains, NMT systems tend to generate fluent translations at the expense of adequacy. We present a stack-based latti…
Domain AdaptationMachine TranslationNMTTranslationNeural Machine Translation by Minimising the Bayes-risk with Respect to Syntactic Translation Lattices
We present a novel scheme to combine neural machine translation (NMT) with traditional statistical machine translation (SMT). Our approach borrows ideas from linearised lattice minimum Bayes-risk decoding for SMT. The NM…
DecoderMachine TranslationNMTTranslationLattention: Lattice-attention in ASR rescoring
Lattices form a compact representation of multiple hypotheses generated from an automatic speech recognition system and have been shown to improve performance of downstream tasks like spoken language understanding and sp…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Decoderspeech-recognition+3Sequence-to-Lattice Models for Fast Translation
Non-autoregressive machine translation (NAT) approaches enable fast generation by utilizing parallelizable generative processes. The remaining bottleneck in these models is their decoder layers; unfortunately unlike in a…
DecoderMachine TranslationTranslation