Plan, Attend, Generate: Character-level Neural Machine Translation with Planning in the Decoder

2017-06-13 · Caglar Gulcehre, Francis Dutil, Adam Trischler, Yoshua Bengio

We investigate the integration of a planning mechanism into an encoder-decoder architecture with an explicit alignment for character-level machine translation. We develop a model that plans ahead when it computes alignments between the source and target sequences, constructing a matrix of proposed future alignments and a commitment vector that governs whether to follow or recompute the plan. This mechanism is inspired by the strategic attentive reader and writer (STRAW) model. Our proposed model is end-to-end trainable with fully differentiable operations. We show that it outperforms a strong baseline on three character-level decoder neural machine translation on WMT'15 corpus. Our analysis demonstrates that our model can compute qualitatively intuitive alignments and achieves superior performance with fewer parameters.

📄 PDF Abstract BibTeX arXiv:1706.05087

Code (1)

nyu-dl/dl4mt-cdec 공식 구현

Tasks

Decoder Machine Translation Translation

Plan, Attend, Generate: Character-Level Neural Machine Translation with Planning

2017-08-01 · WS 2017 8 · Caglar Gulcehre, Francis Dutil, Adam Trischler, Yoshua Bengio

We investigate the integration of a planning mechanism into an encoder-decoder architecture with attention. We develop a model that can plan ahead when it computes alignments between the source and target sequences not o…

Decoder Hierarchical Reinforcement Learning Machine Translation Reinforcement Learning+3

Plan, Attend, Generate: Planning for Sequence-to-Sequence Models

2017-11-28 · NeurIPS 2017 12 · Francis Dutil, Caglar Gulcehre, Adam Trischler, Yoshua Bengio

We investigate the integration of a planning mechanism into sequence-to-sequence models using attention. We develop a model which can plan ahead in the future when it computes its alignments between input and output sequ…

Question Generation Question-Generation Reinforcement Learning Reinforcement Learning (RL)+1

Effective Character-augmented Word Embedding for Machine Reading Comprehension

2018-08-07 · Zhuosheng Zhang, Yafang Huang, Pengfei Zhu, Hai Zhao

Machine reading comprehension is a task to model relationship between passage and query. In terms of deep learning framework, most of state-of-the-art models simply concatenate word and character level representations, w…

Machine Reading Comprehension Reading Comprehension

Area Attention

2018-10-23 · ICLR 2019 5 · Yang Li, Lukasz Kaiser, Samy Bengio, Si Si

Existing attention mechanisms are trained to attend to individual items in a collection (the memory) with a predefined, fixed granularity, e.g., a word token or an image grid. We propose area attention: a way to attend t…

Image Captioning Machine Translation Translation

Attending Form and Context to Generate Specialized Out-of-VocabularyWords Representations

2019-12-14 · Nicolas Garneau, Jean-Samuel Leboeuf, Yuval Pinter, Luc Lamontagne

We propose a new contextual-compositional neural network layer that handles out-of-vocabulary (OOV) words in natural language processing (NLP) tagging tasks. This layer consists of a model that attends to both the charac…

Form Sentence