Recurrent Neural Network Grammars
We introduce recurrent neural network grammars, probabilistic models of sentences with explicit phrase structure. We explain efficient inference procedures that allow application to both parsing and language modeling. Experiments show that they provide better parsing in English than any single previously published supervised generative model and better language modeling than state-of-the-art sequential RNNs in English and Chinese.
Code (6)
Tasks
Constituency ParsingLanguage ModelingLanguage ModellingSimilar Papers 제목 키워드 기반
Connecting First and Second Order Recurrent Networks with Deterministic Finite Automata
We propose an approach that connects recurrent networks with different orders of hidden interaction with regular grammars of different levels of complexity. We argue that the correspondence between recurrent networks and…
AllA Comparative Study of Rule Extraction for Recurrent Neural Networks
Understanding recurrent networks through rule extraction has a long history. This has taken on new interests due to the need for interpreting or verifying neural networks. One basic form for representing stateful rules i…
Learning Argument Structures with Recurrent Neural Network Grammars
Verification of Recurrent Neural Networks Through Rule Extraction
The verification problem for neural networks is verifying whether a neural network will suffer from adversarial samples, or approximating the maximal allowed scale of adversarial perturbation that can be endured. While m…
What Do Recurrent Neural Network Grammars Learn About Syntax?
Recurrent neural network grammars (RNNG) are a recently proposed probabilistic generative modeling family for natural language. They show state-of-the-art language modeling and parsing performance. We investigate what in…
Constituency ParsingDependency ParsingLanguage ModelingLanguage Modelling