Exploring Contextualized Neural Language Models for Temporal Dependency Parsing

2020-04-30 · EMNLP 2020 11 · Hayley Ross, Jonathon Cai, Bonan Min

Extracting temporal relations between events and time expressions has many applications such as constructing event timelines and time-related question answering. It is a challenging problem which requires syntactic and semantic information at sentence or discourse levels, which may be captured by deep contextualized language models (LMs) such as BERT (Devlin et al., 2019). In this paper, we develop several variants of BERT-based temporal dependency parser, and show that BERT significantly improves temporal dependency parsing (Zhang and Xue, 2018a). We also present a detailed analysis on why deep contextualized neural LMs help and where they may fall short. Source code and resources are made available at https://github.com/bnmin/tdp_ranking.

📄 PDF Abstract BibTeX arXiv:2004.14577

Code (1)

bnmin/tdp_ranking 공식 구현 tf

Tasks

Dependency Parsing Question AnsweringSentence

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Residual Connection 설명 없음

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Weight Decay 설명 없음

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Adam 설명 없음

Evaluating Contextualized Embeddings on 54 Languages in POS Tagging, Lemmatization and Dependency Parsing

2019-08-20 · Milan Straka, Jana Straková, Jan Hajič

We present an extensive evaluation of three recently proposed methods for contextualized embeddings on 89 corpora in 54 languages of the Universal Dependencies 2.3 in three tasks: POS tagging, lemmatization, and dependen…

Dependency Parsing Lemmatization POS POS Tagging+1

A Joint Deep Contextualized Word Representation for Deep Biaffine Dependency Parsing

2020-12-01 · VLSP 2020 12 · Xuan-Dung Doan

Dependency Parsing

Deep Contextualized Self-training for Low Resource Dependency Parsing

2019-11-11 · TACL 2019 3 · Guy Rotman, Roi Reichart

Neural dependency parsing has proven very effective, achieving state-of-the-art results on numerous domains and languages. Unfortunately, it requires large amounts of labeled data, that is costly and laborious to create.…

Dependency Parsing

Semi-supervised Domain Adaptation for Dependency Parsing via Improved Contextualized Word Representations

2020-12-01 · COLING 2020 8 · Ying Li, Zhenghua Li, Min Zhang

In recent years, parsing performance is dramatically improved on in-domain texts thanks to the rapid progress of deep neural network models. The major challenge for current parsing research is to improve parsing performa…

Dependency Parsing Domain Adaptation Language Modelling Semi-supervised Domain Adaptation

Dependency Parsing with Bottom-up Hierarchical Pointer Networks

2021-05-20 · Daniel Fernández-González, Carlos Gómez-Rodríguez

Dependency parsing is a crucial step towards deep language understanding and, therefore, widely demanded by numerous Natural Language Processing applications. In particular, left-to-right and top-down transition-based al…

Dependency Parsing Sentence