Compositionality for Recursive Neural Networks
Modelling compositionality has been a longstanding area of research in the field of vector space semantics. The categorical approach to compositionality maps grammar onto vector spaces in a principled way, but comes under fire for requiring the formation of very high-dimensional matrices and tensors, and therefore being computationally infeasible. In this paper I show how a linear simplification of recursive neural tensor network models can be mapped directly onto the categorical approach, giving a way of computing the required matrices and tensors. This mapping suggests a number of lines of research for both categorical compositional vector space models of meaning and for recursive neural network models of compositionality.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Recursive Neural Networks with Bottlenecks Diagnose (Non-)Compositionality
A recent line of work in NLP focuses on the (dis)ability of models to generalise compositionally for artificial languages. However, when considering natural language tasks, the data involved is not strictly, or locally, …
Sentiment AnalysisSentiment ClassificationSemantic Compositionality through Recursive Matrix-Vector Spaces
Deep Recursive Neural Networks for Compositionality in Language
Recursive neural networks comprise a class of architecture that can operate on structured input. They have been previously successfully applied to model compositionality in natural language using parse-tree-based structu…
Sentiment AnalysisSentiment ClassificationRecursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Dynamic Compositionality in Recursive Neural Networks with Structure-aware Tag Representations
Most existing recursive neural network (RvNN) architectures utilize only the structure of parse trees, ignoring syntactic tags which are provided as by-products of parsing. We present a novel RvNN architecture that can p…
Natural Language InferenceSentenceSentiment AnalysisTAG