Plan, Attend, Generate: Character-Level Neural Machine Translation with Planning

2017-08-01 · WS 2017 8 · Caglar Gulcehre, Francis Dutil, Adam Trischler, Yoshua Bengio

We investigate the integration of a planning mechanism into an encoder-decoder architecture with attention. We develop a model that can plan ahead when it computes alignments between the source and target sequences not only for a single time-step but for the next k time-steps as well by constructing a matrix of proposed future alignments and a commitment vector that governs whether to follow or recompute the plan. This mechanism is inspired by strategic attentive reader and writer (STRAW) model, a recent neural architecture for planning with hierarchical reinforcement learning that can also learn higher level temporal abstractions. Our proposed model is end-to-end trainable with differentiable operations. We show that our model outperforms strong baselines on character-level translation task from WMT{'}15 with fewer parameters and computes alignments that are qualitatively intuitive.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Decoder Hierarchical Reinforcement Learning Machine Translation Reinforcement Learning Reinforcement Learning (RL)Representation Learning Translation

Plan, Attend, Generate: Character-level Neural Machine Translation with Planning in the Decoder

2017-06-13 · Caglar Gulcehre, Francis Dutil, Adam Trischler, Yoshua Bengio

We investigate the integration of a planning mechanism into an encoder-decoder architecture with an explicit alignment for character-level machine translation. We develop a model that plans ahead when it computes alignme…

Decoder Machine Translation Translation

Plan, Attend, Generate: Planning for Sequence-to-Sequence Models

2017-11-28 · NeurIPS 2017 12 · Francis Dutil, Caglar Gulcehre, Adam Trischler, Yoshua Bengio

We investigate the integration of a planning mechanism into sequence-to-sequence models using attention. We develop a model which can plan ahead in the future when it computes its alignments between input and output sequ…

Question Generation Question-Generation Reinforcement Learning Reinforcement Learning (RL)+1

Effective Character-augmented Word Embedding for Machine Reading Comprehension

2018-08-07 · Zhuosheng Zhang, Yafang Huang, Pengfei Zhu, Hai Zhao

Machine reading comprehension is a task to model relationship between passage and query. In terms of deep learning framework, most of state-of-the-art models simply concatenate word and character level representations, w…

Machine Reading Comprehension Reading Comprehension

Area Attention

2018-10-23 · ICLR 2019 5 · Yang Li, Lukasz Kaiser, Samy Bengio, Si Si

Existing attention mechanisms are trained to attend to individual items in a collection (the memory) with a predefined, fixed granularity, e.g., a word token or an image grid. We propose area attention: a way to attend t…

Image Captioning Machine Translation Translation

Attending Form and Context to Generate Specialized Out-of-VocabularyWords Representations

2019-12-14 · Nicolas Garneau, Jean-Samuel Leboeuf, Yuval Pinter, Luc Lamontagne

We propose a new contextual-compositional neural network layer that handles out-of-vocabulary (OOV) words in natural language processing (NLP) tagging tasks. This layer consists of a model that attends to both the charac…

Form Sentence