Dynamic Compositionality in Recursive Neural Networks with Structure-aware Tag Representations
Most existing recursive neural network (RvNN) architectures utilize only the structure of parse trees, ignoring syntactic tags which are provided as by-products of parsing. We present a novel RvNN architecture that can provide dynamic compositionality by considering comprehensive syntactic information derived from both the structure and linguistic tags. Specifically, we introduce a structure-aware tag representation constructed by a separate tag-level tree-LSTM. With this, we can control the composition function of the existing word-level tree-LSTM by augmenting the representation as a supplementary input to the gate functions of the tree-LSTM. In extensive experiments, we show that models built upon the proposed architecture obtain superior or competitive performance on several sentence-level tasks such as sentiment analysis and natural language inference when compared against previous tree-structured models and other sophisticated neural models.
Code (2)
Tasks
Natural Language InferenceSentenceSentiment AnalysisTAGSimilar Papers 제목 키워드 기반
Deep Recursive Neural Networks for Compositionality in Language
Recursive neural networks comprise a class of architecture that can operate on structured input. They have been previously successfully applied to model compositionality in natural language using parse-tree-based structu…
Sentiment AnalysisSentiment ClassificationDynamic Compositional Neural Networks over Tree Structure
Tree-structured neural networks have proven to be effective in learning semantic representations by exploiting syntactic information. In spite of their success, most existing models suffer from the underfitting problem: …
Learning Semantic RepresentationsRecursive Neural Networks with Bottlenecks Diagnose (Non-)Compositionality
A recent line of work in NLP focuses on the (dis)ability of models to generalise compositionally for artificial languages. However, when considering natural language tasks, the data involved is not strictly, or locally, …
Sentiment AnalysisSentiment ClassificationCompositionality for Recursive Neural Networks
Modelling compositionality has been a longstanding area of research in the field of vector space semantics. The categorical approach to compositionality maps grammar onto vector spaces in a principled way, but comes unde…
Compositionality-Aware Graph2Seq Learning
Graphs are a highly expressive data structure, but it is often difficult for humans to find patterns from a complex graph. Hence, generating human-interpretable sequences from graphs have gained interest, called graph2se…
Code SummarizationSource Code Summarization