Long Short-Term Memory Over Tree Structures
The chain-structured long short-term memory (LSTM) has showed to be effective in a wide range of problems such as speech recognition and machine translation. In this paper, we propose to extend it to tree structures, in which a memory cell can reflect the history memories of multiple child cells or multiple descendant cells in a recursive process. We call the model S-LSTM, which provides a principled way of considering long-distance interaction over hierarchies, e.g., language or image parse structures. We leverage the models for semantic composition to understand the meaning of text, a fundamental problem in natural language understanding, and show that it outperforms a state-of-the-art recursive model by replacing its composition layers with the S-LSTM memory blocks. We also show that utilizing the given structures is helpful in achieving a performance better than that without considering the structures.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationNatural Language UnderstandingSemantic Compositionspeech-recognitionSpeech RecognitionTranslationSimilar Papers 제목 키워드 기반
Inducing Grammar from Long Short-Term Memory Networks by Shapley Decomposition
The principle of compositionality has deep roots in linguistics: the meaning of an expression is determined by its structure and the meanings of its constituents. However, modern neural network models such as long short-…
Top-down Tree Long Short-Term Memory Networks
Long Short-Term Memory (LSTM) networks, a type of recurrent neural network with a more complex computational unit, have been successfully applied to a variety of sequence modeling tasks. In this paper we develop Tree Lon…
Dependency ParsingRerankingSentenceSentence CompletionCompositional Distributional Semantics with Long Short Term Memory
We are proposing an extension of the recursive neural network that makes use of a variant of the long short-term memory architecture. The extension allows information low in parse trees to be stored in a memory register …
Tree Memory Networks for Modelling Long-term Temporal Dependencies
In the domain of sequence modelling, Recurrent Neural Networks (RNN) have been capable of achieving impressive results in a variety of application areas including visual question answering, part-of-speech tagging and mac…
Machine TranslationPart-Of-Speech TaggingQuestion AnsweringTrajectory Prediction+2Japanese Sentiment Classification using a Tree-Structured Long Short-Term Memory with Attention
Previous approaches to training syntax-based sentiment classification models required phrase-level annotated corpora, which are not readily available in many languages other than English. Thus, we propose the use of tree…
ClassificationGeneral ClassificationSentiment AnalysisSentiment Classification