Recurrent Graph Syntax Encoder for Neural Machine Translation
Syntax-incorporated machine translation models have been proven successful in improving the model's reasoning and meaning preservation ability. In this paper, we propose a simple yet effective graph-structured encoder, the Recurrent Graph Syntax Encoder, dubbed \textbf{RGSE}, which enhances the ability to capture useful syntactic information. The RGSE is done over a standard encoder (recurrent or self-attention encoder), regarding recurrent network units as graph nodes and injects syntactic dependencies as edges, such that RGSE models syntactic dependencies and sequential information (\textit{i.e.}, word order) simultaneously. Our approach achieves considerable improvements over several syntax-aware NMT models in English$\Rightarrow$German and English$\Rightarrow$Czech translation tasks. And RGSE-equipped big model obtains competitive result compared with the state-of-the-art model in WMT14 En-De task. Extensive analysis further verifies that RGSE could benefit long sentence modeling, and produces better translations.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationNMTSentenceTranslationSimilar Papers 제목 키워드 기반
Top-down Tree Structured Decoding with Syntactic Connections for Neural Machine Translation and Parsing
The addition of syntax-aware decoding in Neural Machine Translation (NMT) systems requires an effective tree-structured neural network, a syntax-aware attention model and a language generation model that is sensitive to …
Constituency ParsingDecoderDependency ParsingMachine Translation+4Graph Convolutional Encoders for Syntax-aware Neural Machine Translation
We present a simple and effective approach to incorporating syntactic structure into neural attention-based encoder-decoder models for machine translation. We rely on graph-convolutional networks (GCNs), a recent class o…
DecoderMachine TranslationTranslationIncorporating Source Syntax into Transformer-Based Neural Machine Translation
Transformer-based neural machine translation (NMT) has recently achieved state-of-the-art performance on many machine translation tasks. However, recent work (Raganato and Tiedemann, 2018; Tang et al., 2018; Tran et al.,…
DecoderMachine TranslationNMTTranslationNeural Machine Translation with Dynamic Graph Convolutional Decoder
Existing wisdom demonstrates the significance of syntactic knowledge for the improvement of neural machine translation models. However, most previous works merely focus on leveraging the source syntax in the well-known e…
DecoderMachine TranslationTranslationAn Intrinsic Nearest Neighbor Analysis of Neural Machine Translation Architectures
Earlier approaches indirectly studied the information captured by the hidden states of recurrent and non-recurrent neural machine translation models by feeding them into different classifiers. In this paper, we look at t…
Machine TranslationTranslationWord Embeddings