Refining Source Representations with Relation Networks for Neural Machine Translation
Although neural machine translation (NMT) with the encoder-decoder framework has achieved great success in recent times, it still suffers from some drawbacks: RNNs tend to forget old information which is often useful and the encoder only operates through words without considering word relationship. To solve these problems, we introduce a relation networks (RN) into NMT to refine the encoding representations of the source. In our method, the RN first augments the representation of each source word with its neighbors and reasons all the possible pairwise relations between them. Then the source representations and all the relations are fed to the attention module and the decoder together, keeping the main encoder-decoder architecture unchanged. Experiments on two Chinese-to-English data sets in different scales both show that our method can outperform the competitive baselines significantly.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderMachine TranslationNMTRelationTranslationSimilar Papers 제목 키워드 기반
Refining Source Representations with Relation Networks for Neural Machine Translation
Although neural machine translation with the encoder-decoder framework has achieved great success recently, it still suffers drawbacks of forgetting distant information, which is an inherent disadvantage of recurrent neu…
DecoderMachine TranslationMemorizationRelation+1Learning to Refine Source Representations for Neural Machine Translation
Neural machine translation (NMT) models generally adopt an encoder-decoder architecture for modeling the entire translation process. The encoder summarizes the representation of input sentence from scratch, which is pote…
DecoderMachine TranslationNMTReinforcement Learning+2Neural Machine Translation with Joint Representation
Though early successes of Statistical Machine Translation (SMT) systems are attributed in part to the explicit modelling of the interaction between any two source and target units, e.g., alignment, the recent Neural Mach…
DecoderMachine TranslationNMTTranslationBridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations
Sparse language vectors from linguistic typology databases and learned embeddings from tasks like multilingual machine translation have been investigated in isolation, without analysing how they could benefit from each o…
ClusteringMachine TranslationTranslationLearning Source Phrase Representations for Neural Machine Translation
The Transformer translation model (Vaswani et al., 2017) based on a multi-head attention mechanism can be computed effectively in parallel and has significantly pushed forward the performance of Neural Machine Translatio…
Machine TranslationNMTTranslation