Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs
We introduce a neural network that represents sentences by composing their words according to induced binary parse trees. We use Tree-LSTM as our composition function, applied along a tree structure found by a fully differentiable natural language chart parser. Our model simultaneously optimises both the composition function and the parser, thus eliminating the need for externally-provided parse trees which are normally required for Tree-LSTM. It can therefore be seen as a tree-based RNN that is unsupervised with respect to the parse trees. As it is fully differentiable, our model is easily trained with an off-the-shelf gradient descent method and backpropagation. We demonstrate that it achieves better performance compared to various supervised Tree-LSTM architectures on a textual entailment task and a reverse dictionary task.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language InferenceReverse DictionarySentenceSentence EmbeddingsSimilar Papers 제목 키워드 기반
Disentangling Semantics and Syntax in Sentence Embeddings with Pre-trained Language Models
Pre-trained language models have achieved huge success on a wide range of NLP tasks. However, contextual representations from pre-trained models contain entangled semantic and syntactic information, and therefore cannot …
Semantic SimilaritySemantic Textual SimilaritySentenceSentence Embedding+2Unsupervised Recurrent Neural Network Grammars
Recurrent neural network grammars (RNNG) are generative models of language which jointly model syntax and surface structure by incrementally generating a syntax tree and sentence in a top-down, left-to-right order. Super…
Constituency Grammar InductionLanguage ModelingLanguage ModellingSentence+1Syntax Encoding with Application in Authorship Attribution
We propose a novel strategy to encode the syntax parse tree of sentence into a learnable distributed representation. The proposed syntax encoding scheme is provably information-lossless. In specific, an embedding vector …
Authorship AttributionBenchmarkingFeature EngineeringSentence+1Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders
We introduce the deep inside-outside recursive autoencoder (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. Our approach pre…
Constituency Grammar InductionConstituency ParsingSentenceJoint Unsupervised Learning of Semantic Representation of Words and Roles in Dependency Trees
In this paper, we introduce WoRel, a model that jointly learns word embeddings and a semantic representation of word relations. The model learns from plain text sentences and their dependency parse trees. The word embedd…
Named Entity Recognition (NER)Question AnsweringSentenceSentiment Analysis+2