An Imitation Learning Approach to Unsupervised Parsing
Recently, there has been an increasing interest in unsupervised parsers that optimize semantically oriented objectives, typically using reinforcement learning. Unfortunately, the learned trees often do not match actual syntax trees well. Shen et al. (2018) propose a structured attention mechanism for language modeling (PRPN), which induces better syntactic structures but relies on ad hoc heuristics. Also, their model lacks interpretability as it is not grounded in parsing actions. In our work, we propose an imitation learning approach to unsupervised parsing, where we transfer the syntactic knowledge induced by the PRPN to a Tree-LSTM model with discrete parsing actions. Its policy is then refined by Gumbel-Softmax training towards a semantically oriented objective. We evaluate our approach on the All Natural Language Inference dataset and show that it achieves a new state of the art in terms of parsing $F$-score, outperforming our base models, including the PRPN.
Code (1)
Tasks
Imitation LearningLanguage ModelingLanguage ModellingNatural Language InferenceReinforcement LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Adapting Unsupervised Syntactic Parsing Methodology for Discourse Dependency Parsing
One of the main bottlenecks in developing discourse dependency parsers is the lack of annotated training data. A potential solution is to utilize abundant unlabeled data by using unsupervised techniques, but there is so …
Dependency ParsingDiscourse ParsingOn the Role of Supervision in Unsupervised Constituency Parsing
We analyze several recent unsupervised constituency parsing models, which are tuned with respect to the parsing $F_1$ score on the Wall Street Journal (WSJ) development set (1,700 sentences). We introduce strong baseline…
Constituency ParsingData AugmentationModel SelectionA Survey of Unsupervised Dependency Parsing
Syntactic dependency parsing is an important task in natural language processing. Unsupervised dependency parsing aims to learn a dependency parser from sentences that have no annotation of their correct parse trees. Des…
Dependency ParsingSurveyUnsupervised Dependency ParsingUnsupervised Inference of Data-Driven Discourse Structures using a Tree Auto-Encoder
With a growing need for robust and general discourse structures in many downstream tasks and real-world applications, the current lack of high-quality, high-quantity discourse trees poses a severe shortcoming. In order t…
Discourse ParsingUnsupervised Dependency Parsing: Let's Use Supervised Parsers
We present a self-training approach to unsupervised dependency parsing that reuses existing supervised and unsupervised parsing algorithms. Our approach, called `iterated reranking' (IR), starts with dependency trees gen…
Dependency ParsingRerankingUnsupervised Dependency Parsing