Towards Neural Machine Translation with Latent Tree Attention
Building models that take advantage of the hierarchical structure of language without a priori annotation is a longstanding goal in natural language processing. We introduce such a model for the task of machine translation, pairing a recurrent neural network grammar encoder with a novel attentional RNNG decoder and applying policy gradient reinforcement learning to induce unsupervised tree structures on both the source and target. When trained on character-level datasets with no explicit segmentation or parse annotation, the model learns a plausible segmentation and shallow parse, obtaining performance close to an attentional baseline.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderMachine Translationreinforcement-learningReinforcement LearningReinforcement Learning (RL)SegmentationTranslationSimilar Papers 제목 키워드 기반
Inducing Constituency Trees through Neural Machine Translation
Latent tree learning(LTL) methods learn to parse sentences using only indirect supervision from a downstream task. Recent advances in latent tree learning have made it possible to recover moderately high quality tree str…
Language ModelingLanguage ModellingMachine TranslationTranslationNeural Machine Translation with Source-Side Latent Graph Parsing
This paper presents a novel neural machine translation model which jointly learns translation and source-side latent graph representations of sentences. Unlike existing pipelined approaches using syntactic parsers, our e…
Machine TranslationNMTTranslationImproved Neural Machine Translation with a Syntax-Aware Encoder and Decoder
Most neural machine translation (NMT) models are based on the sequential encoder-decoder framework, which makes no use of syntactic information. In this paper, we improve this model by explicitly incorporating source-sid…
DecoderMachine TranslationNMTTranslationInteractive Visualization and Manipulation of Attention-based Neural Machine Translation
While neural machine translation (NMT) provides high-quality translation, it is still hard to interpret and analyze its behavior. We present an interactive interface for visualizing and intervening behavior of NMT, speci…
Machine TranslationNMTTranslationSyntax-Based Attention Masking for Neural Machine Translation
We present a simple method for extending transformers to source-side trees. We define a number of masks that limit self-attention based on relationships among tree nodes, and we allow each attention head to learn which m…
Machine TranslationTranslation