Constituency Parsing with a Self-Attentive Encoder
We demonstrate that replacing an LSTM encoder with a self-attentive architecture can lead to improvements to a state-of-the-art discriminative constituency parser. The use of attention makes explicit the manner in which information is propagated between different locations in the sentence, which we use to both analyze our model and propose potential improvements. For example, we find that separating positional and content information in the encoder can lead to improved parsing accuracy. Additionally, we evaluate different approaches for lexical representation. Our parser achieves new state-of-the-art results for single models trained on the Penn Treebank: 93.55 F1 without the use of any external data, and 95.13 F1 when using pre-trained word representations. Our parser also outperforms the previous best-published accuracy figures on 8 of the 9 languages in the SPMRL dataset.
Code (5)
Tasks
Constituency ParsingSentenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Investigating Non-local Features for Neural Constituency Parsing
Thanks to the strong representation power of neural encoders, neural chart-based parsers have achieved highly competitive performance by using local features. Recently, it has been shown that non-local features in CRF st…
Constituency ParsingSelf-Attentive Constituency Parsing for UCCA-based Semantic Parsing
Semantic parsing provides a way to extract the semantic structure of a text that could be understood by machines. It is utilized in various NLP applications that require text comprehension such as summarization and quest…
Constituency ParsingFew-Shot LearningQuestion AnsweringReading Comprehension+1LLM-enhanced Self-training for Cross-domain Constituency Parsing
Self-training has proven to be an effective approach for cross-domain tasks, and in this study, we explore its application to cross-domain constituency parsing. Traditional self-training methods rely on limited and poten…
Constituency ParsingLanguage ModelingLanguage ModellingLarge Language ModelAn Empirical Study for Vietnamese Constituency Parsing with Pre-training
In this work, we use a span-based approach for Vietnamese constituency parsing. Our method follows the self-attention encoder architecture and a chart decoder using a CKY-style inference algorithm. We present analyses of…
Constituency ParsingDecoderVietnamese DatasetsVietnamese ParsingInvestigating Non-local Features for Neural Constituency Parsing
Thanks to the strong representation power of neural encoders, neural chart-based parsers have achieved highly competitive performance by using local features. Recently, it has been shown that non-local features in CRF st…
Constituency Parsing