Recurrent Neural Networks with External Memory for Language Understanding
Recurrent Neural Networks (RNNs) have become increasingly popular for the task of language understanding. In this task, a semantic tagger is deployed to associate a semantic label to each word in an input sequence. The success of RNN may be attributed to its ability to memorize long-term dependence that relates the current-time semantic label prediction to the observations many time instances away. However, the memory capacity of simple RNNs is limited because of the gradient vanishing and exploding problem. We propose to use an external memory to improve memorization capability of RNNs. We conducted experiments on the ATIS dataset, and observed that the proposed model was able to achieve the state-of-the-art results. We compare our proposed model with alternative models and report analysis results that may provide insights for future research.
Code (0)
등록된 구현이 없습니다.
Tasks
MemorizationSimilar Papers 제목 키워드 기반
State-Regularized Recurrent Neural Networks
Recurrent neural networks are a widely used class of neural architectures. They have, however, two shortcomings. First, it is difficult to understand what exactly they learn. Second, they tend to work poorly on sequences…
Language ModelingLanguage ModellingMemorizationObject Recognition+1Recurrent Autoregressive Networks for Online Multi-Object Tracking
The main challenge of online multi-object tracking is to reliably associate object trajectories with detections in each video frame based on their tracking history. In this work, we propose the Recurrent Autoregressive N…
Multi-Object TrackingObjectObject TrackingOnline Multi-Object TrackingRecurrent Memory Networks for Language Modeling
Recurrent Neural Networks (RNN) have obtained excellent result in many natural language processing (NLP) tasks. However, understanding and interpreting the source of this success remains a challenge. In this paper, we pr…
Language ModelingLanguage ModellingSentenceSentence CompletionRecognizing Long Grammatical Sequences Using Recurrent Networks Augmented With An External Differentiable Stack
Recurrent neural networks (RNNs) are a widely used deep architecture for sequence modeling, generation, and prediction. Despite success in applications such as machine translation and voice recognition, these stateful mo…
Language ModellingMachine TranslationTime Series AnalysisTime Series ForecastingTransformers are Stateless Differentiable Neural Computers
Differentiable Neural Computers (DNCs) were introduced as recurrent architectures equipped with an addressable external memory supporting differentiable read and write operations. Transformers, in contrast, are nominally…