Unsupervised Parsing with S-DIORA: Single Tree Encoding for Deep Inside-Outside Recursive Autoencoders
The deep inside-outside recursive autoencoder (DIORA; Drozdov et al. 2019) is a self-supervised neural model that learns to induce syntactic tree structures for input sentences *without access to labeled training data*. In this paper, we discover that while DIORA exhaustively encodes all possible binary trees of a sentence with a soft dynamic program, its vector averaging approach is locally greedy and cannot recover from errors when computing the highest scoring parse tree in bottom-up chart parsing. To fix this issue, we introduce S-DIORA, an improved variant of DIORA that encodes a single tree rather than a softly-weighted mixture of trees by employing a hard argmax operation and a beam at each cell in the chart. Our experiments show that through *fine-tuning* a pre-trained DIORA with our new algorithm, we improve the state of the art in *unsupervised* constituency parsing on the English WSJ Penn Treebank by 2.2-6{\%} F1, depending on the data used for fine-tuning.
Code (0)
등록된 구현이 없습니다.
Tasks
Constituency Grammar InductionConstituency ParsingSentenceSimilar Papers 제목 키워드 기반
Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders
We introduce the deep inside-outside recursive autoencoder (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. Our approach pre…
Constituency Grammar InductionConstituency ParsingSentenceUnsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoders
We introduce deep inside-outside recursive autoencoders (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. Our approach predic…
Constituency ParsingSentenceImproved Latent Tree Induction with Distant Supervision via Span Constraints
For over thirty years, researchers have developed and analyzed methods for latent tree induction as an approach for unsupervised syntactic parsing. Nonetheless, modern systems still do not perform well enough compared to…
Constituency ParsingRule Augmented Unsupervised Constituency Parsing
Recently, unsupervised parsing of syntactic trees has gained considerable attention. A prototypical approach to such unsupervised parsing employs reinforcement learning and auto-encoders. However, no mechanism ensures th…
Constituency Parsingreinforcement-learningReinforcement Learning (RL)Deep Inside-outside Recursive Autoencoder with All-span Objective
Deep inside-outside recursive autoencoder (DIORA) is a neural-based model designed for unsupervised constituency parsing. During its forward computation, it provides phrase and contextual representations for all spans in…
AllConstituency ParsingSentence