Recurrently Controlled Recurrent Networks
Recurrent neural networks (RNNs) such as long short-term memory and gated recurrent units are pivotal building blocks across a broad spectrum of sequence modeling problems. This paper proposes a recurrently controlled recurrent network (RCRN) for expressive and powerful sequence encoding. More concretely, the key idea behind our approach is to learn the recurrent gating functions using recurrent networks. Our architecture is split into two components - a controller cell and a listener cell whereby the recurrent controller actively influences the compositionality of the listener cell. We conduct extensive experiments on a myriad of tasks in the NLP domain such as sentiment analysis (SST, IMDb, Amazon reviews, etc.), question classification (TREC), entailment classification (SNLI, SciTail), answer selection (WikiQA, TrecQA) and reading comprehension (NarrativeQA). Across all 26 datasets, our results demonstrate that RCRN not only consistently outperforms BiLSTMs but also stacked BiLSTMs, suggesting that our controller architecture might be a suitable replacement for the widely adopted stacked architecture.
Code (1)
Tasks
Answer SelectionGeneral ClassificationReading ComprehensionSentiment AnalysisSimilar Papers 제목 키워드 기반
Recurrently Controlling a Recurrent Network with Recurrent Networks Controlled by More Recurrent Networks
This paper explores an intriguing idea of recursively parameterizing recurrent nets. Simply speaking, this refers to recurrently controlling a recurrent network with recurrent networks controlled by recurrent networks. T…
Code GenerationInductive BiasMachine TranslationMusic Modeling+2Recurrent Stacking of Layers in Neural Networks: An Application to Neural Machine Translation
In deep neural network modeling, the most common practice is to stack a number of recurrent, convolutional, or feed-forward layers in order to obtain high-quality continuous space representations which in turn improves t…
Knowledge DistillationMachine TranslationNMTTransfer Learning+1Recurrent Attention Model with Log-Polar Mapping is Robust against Adversarial Attacks
Convolutional neural networks are vulnerable to small $\ell^p$ adversarial attacks, while the human visual system is not. Inspired by neural networks in the eye and the brain, we developed a novel artificial neural netwo…
Recurrent Stacking of Layers for Compact Neural Machine Translation Models
In neural machine translation (NMT), the most common practice is to stack a number of recurrent or feed-forward layers in the encoder and the decoder. As a result, the addition of each new layer improves the translation …
DecoderMachine TranslationNMTTranslation