Recursive Subtree Composition in LSTM-Based Dependency Parsing
The need for tree structure modelling on top of sequence modelling is an open issue in neural dependency parsing. We investigate the impact of adding a tree layer on top of a sequential model by recursively composing subtree representations (composition) in a transition-based parser that uses features extracted by a BiLSTM. Composition seems superfluous with such a model, suggesting that BiLSTMs capture information about subtrees. We perform model ablations to tease out the conditions under which composition helps. When ablating the backward LSTM, performance drops and composition does not recover much of the gap. When ablating the forward LSTM, performance drops less dramatically and composition recovers a substantial part of the gap, indicating that a forward LSTM and composition capture similar information. We take the backward LSTM to be related to lookahead features and the forward LSTM to the rich history-based features both crucial for transition-based parsers. To capture history-based information, composition is better than a forward LSTM on its own, but it is even better to have a forward LSTM as part of a BiLSTM. We correlate results with language properties, showing that the improved lookahead of a backward LSTM is especially important for head-final languages.
Code (1)
Tasks
Dependency ParsingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Effective Representation for Easy-First Dependency Parsing
Easy-first parsing relies on subtree re-ranking to build the complete parse tree. Whereas the intermediate state of parsing processing is represented by various subtrees, whose internal structural information is the key …
Dependency ParsingLanguage ModelingLanguage ModellingRe-RankingA subtree-based factorization of dependency parsing
We propose a dependency parsing pipeline, in which the parsing of long-distance projections and localized dependencies are explicitly decomposed at the input level. A chosen baseline dependency parsing model performs onl…
Dependency ParsingIBM Research at the CoNLL 2018 Shared Task on Multilingual Parsing
This paper presents the IBM Research AI submission to the CoNLL 2018 Shared Task on Parsing Universal Dependencies. Our system implements a new joint transition-based parser, based on the Stack-LSTM framework and the Arc…
ARCDependency ParsingMorphological TaggingPart-Of-Speech Tagging+2Dependency Parsing as MRC-based Span-Span Prediction
Higher-order methods for dependency parsing can partially but not fully address the issue that edges in dependency trees should be constructed at the text span/subtree level rather than word level. In this paper, we prop…
Dependency ParsingMachine Reading ComprehensionPredictionReading ComprehensionEasy-First Dependency Parsing with Hierarchical Tree LSTMs
We suggest a compositional vector representation of parse trees that relies on a recursive combination of recurrent-neural network encoders. To demonstrate its effectiveness, we use the representation as the backbone of …
Dependency ParsingWord Embeddings