Second-Order Neural Dependency Parsing with Message Passing and End-to-End Training

2020-10-10 · Asian Chapter of the Association for Computational Linguistics 2020 · Xinyu Wang, Kewei Tu

In this paper, we propose second-order graph-based neural dependency parsing using message passing and end-to-end neural networks. We empirically show that our approaches match the accuracy of very recent state-of-the-art second-order graph-based neural dependency parsers and have significantly faster speed in both training and testing. We also empirically show the advantage of second-order parsing over first-order parsing and observe that the usefulness of the head-selection structured constraint vanishes when using BERT embedding.

📄 PDF Abstract BibTeX arXiv:2010.05003

Code (1)

wangxinyu0922/Second_Order_Parsing 공식 구현 pytorch

Tasks

Dependency Parsing

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Adam 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

WordPiece 설명 없음

Multi-Head Attention 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Combining (second-order) graph-based and headed-span-based projective dependency parsing

2021-08-12 · Findings (ACL) 2022 5 · Songlin Yang, Kewei Tu

Graph-based methods, which decompose the score of a dependency tree into scores of dependency arcs, are popular in dependency parsing for decades. Recently, \citet{Yang2022Span} propose a headed-span-based method that de…

ARC Dependency Parsing

Second-Order Unsupervised Neural Dependency Parsing

2020-10-28 · COLING 2020 8 · Songlin Yang, Yong Jiang, Wenjuan Han, Kewei Tu

Most of the unsupervised dependency parsers are based on first-order probabilistic generative models that only consider local parent-child information. Inspired by second-order supervised dependency parsing, we proposed …

Dependency Grammar Induction Dependency Parsing

Second-Order Semantic Dependency Parsing with End-to-End Neural Networks

2019-06-19 · ACL 2019 7 · Xinyu Wang, Jingxian Huang, Kewei Tu

Semantic dependency parsing aims to identify semantic relationships between words in a sentence that form a graph. In this paper, we propose a second-order semantic dependency parser, which takes into consideration not o…

Dependency Parsing Semantic Dependency Parsing Sentence Variational Inference

Sparse Fuzzy Attention for Structured Sentiment Analysis

2021-09-14 · Letian Peng, Zuchao Li, Hai Zhao

Attention scorers have achieved success in parsing tasks like semantic and syntactic dependency parsing. However, in tasks modeled into parsing, like structured sentiment analysis, "dependency edges" are very sparse whic…

Dependency Parsing Sentiment Analysis

Web-scale Surface and Syntactic n-gram Features for Dependency Parsing

2015-02-25 · Dominick Ng, Mohit Bansal, James R. Curran

We develop novel first- and second-order features for dependency parsing based on the Google Syntactic Ngrams corpus, a collection of subtree counts of parsed sentences from scanned books. We also extend previous work on…