PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols
Probabilistic context-free grammars (PCFGs) with neural parameterization have been shown to be effective in unsupervised phrase-structure grammar induction. However, due to the cubic computational complexity of PCFG representation and parsing, previous approaches cannot scale up to a relatively large number of (nonterminal and preterminal) symbols. In this work, we present a new parameterization form of PCFGs based on tensor decomposition, which has at most quadratic computational complexity in the symbol number and therefore allows us to use a much larger number of symbols. We further use neural parameterization for the new form to improve unsupervised parsing performance. We evaluate our model across ten languages and empirically demonstrate the effectiveness of using more symbols. Our code: https://github.com/sustcsonglin/TN-PCFG
Code (1)
Tasks
Constituency Grammar InductionFormSimilar Papers 제목 키워드 기반
Discriminative Learning for Probabilistic Context-Free Grammars based on Generalized H-Criterion
We present a formal framework for the development of a family of discriminative learning algorithms for Probabilistic Context-Free Grammars (PCFGs) based on a generalization of criterion-H. First of all, we propose the H…
Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs
Hidden Markov Models (HMMs) and Probabilistic Context-Free Grammars (PCFGs) are widely used structured models, both of which can be represented as factor graph grammars (FGGs), a powerful formalism capable of describing …
Language ModelingLanguage ModellingDynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs
Hidden Markov Models (HMMs) and Probabilistic Context-Free Grammars (PCFGs) are widely used structured models, both of which can be represented as factor graph grammars (FGGs), a powerful formalism capable of describing …
Constituency Grammar InductionLanguage ModelingLanguage ModellingConsistent Unsupervised Estimators for Anchored PCFGs
Learning probabilistic context-free grammars (PCFGs) from strings is a classic problem in computational linguistics since Horning (1969). Here we present an algorithm based on distributional learning that is a consistent…
An Empirical Study of Compound PCFGs
Compound probabilistic context-free grammars (C-PCFGs) have recently established a new state of the art for unsupervised phrase-structure grammar induction. However, due to the high space and time complexities of chart-b…
Sentence