Recursive Top-Down Production for Sentence Generation with Latent Trees
We model the recursive production property of context-free grammars for natural and synthetic languages. To this end, we present a dynamic programming algorithm that marginalises over latent binary tree structures with $N$ leaves, allowing us to compute the likelihood of a sequence of $N$ tokens under a latent tree model, which we maximise to train a recursive neural function. We demonstrate performance on two synthetic tasks: SCAN (Lake and Baroni, 2017), where it outperforms previous models on the LENGTH split, and English question formation (McCoy et al., 2020), where it performs comparably to decoders with the ground-truth tree structure. We also present experimental results on German-English translation on the Multi30k dataset (Elliott et al., 2016), and qualitatively analyse the induced tree structures our model learns for the SCAN tasks and the German-English translation task.
Code (1)
Tasks
SentenceTranslationSimilar Papers 제목 키워드 기반
Learning to Embed Sentences Using Attentive Recursive Trees
Sentence embedding is an effective feature representation for most deep learning-based NLP tasks. One prevailing line of methods is using recursive latent tree-structured networks to embed sentences with task-specific st…
SentenceSentence EmbeddingSentence-EmbeddingRocNet: Recursive Octree Network for Efficient 3D Deep Representation
We introduce a deep recursive octree network for the compression of 3D voxel data. Our network compresses a voxel grid of any size down to a very small latent space in an autoencoder-like network. We show results for com…
3D Reconstruction3D Shape Classification3D Shape ReconstructionUnsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders
We introduce the deep inside-outside recursive autoencoder (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. Our approach pre…
Constituency Grammar InductionConstituency ParsingSentenceUnsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoders
We introduce deep inside-outside recursive autoencoders (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. Our approach predic…
Constituency ParsingSentenceUnsupervised Sentence Textual Similarity with Compositional Phrase Semantics
Measuring Sentence Textual Similarity (STS) is a classic task that can be applied to many downstream NLP applications such as text generation and retrieval. In this paper, we focus on unsupervised STS that works on vario…
RetrievalSentenceSentence EmbeddingsSTS+1