홈 › Papers

Transition-based Parsing with Stack-Transformers

2020-10-20 · Findings of the Association for Computational Linguistics 2020 · Ramon Fernandez Astudillo, Miguel Ballesteros, Tahira Naseem, Austin Blodgett, Radu Florian

Modeling the parser state is key to good performance in transition-based parsing. Recurrent Neural Networks considerably improved the performance of transition-based systems by modelling the global state, e.g. stack-LSTM parsers, or local state modeling of contextualized features, e.g. Bi-LSTM parsers. Given the success of Transformer architectures in recent parsing systems, this work explores modifications of the sequence-to-sequence Transformer architecture to model either global or local parser states in transition-based parsing. We show that modifications of the cross attention mechanism of the Transformer considerably strengthen performance both on dependency and Abstract Meaning Representation (AMR) parsing tasks, particularly for smaller models or limited training data.

📄 PDF Abstract BibTeX arXiv:2010.10669

Code (1)

IBM/transition-amr-parser 공식 구현 pytorch

Tasks

Abstract Meaning RepresentationAMR Parsing

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Adam 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Shift-Reduce Task-Oriented Semantic Parsing with Stack-Transformers

2022-10-21 · Daniel Fernández-González

Intelligent voice assistants, such as Apple Siri and Amazon Alexa, are widely used nowadays. These task-oriented dialogue systems require a semantic parsing module in order to process user utterances and understand the a…

Constituency Parsing Semantic Parsing slot-filling Slot Filling+1

Transition-Based Dependency Parsing with Stack Long Short-Term Memory

2015-05-29 · IJCNLP 2015 7 · Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews 외

We propose a technique for learning representations of parser states in transition-based dependency parsers. Our primary innovation is a new control structure for sequence-to-sequence neural networks---the stack LSTM. Li…

Dependency Parsing Transition-Based Dependency Parsing

Transition-based DRS Parsing Using Stack-LSTMs

2019-05-01 · WS 2019 5 · Kilian Evang

We present our submission to the IWCS 2019 shared task on semantic parsing, a transition-based parser that uses explicit word-meaning pairings, but no explicit representation of syntax. Parsing decisions are made based o…

DRS Parsing Semantic Parsing

Stack-based Multi-layer Attention for Transition-based Dependency Parsing

2017-09-01 · EMNLP 2017 9 · Zhirui Zhang, Shujie Liu, Mu Li, Ming Zhou 외

Although sequence-to-sequence (seq2seq) network has achieved significant success in many NLP tasks such as machine translation and text summarization, simply applying this approach to transition-based dependency parsing …

Dependency Parsing Language Modeling Language Modelling Machine Translation+3

Discontinuous Constituency Parsing with a Stack-Free Transition System and a Dynamic Oracle

2019-04-01 · NAACL 2019 6 · Maximin Coavoux, Shay B. Cohen

We introduce a novel transition system for discontinuous constituency parsing. Instead of storing subtrees in a stack --i.e. a data structure with linear-time sequential access-- the proposed system uses a set of parsing…

Constituency Parsing Sentence