MTLB-STRUCT @PARSEME 2020: Capturing Unseen Multiword Expressions Using Multi-task Learning and Pre-trained Masked Language Models

2020-11-04 · COLING (MWE) 2020 12 · Shiva Taslimipoor, Sara Bahaadini, Ekaterina Kochmar

This paper describes a semi-supervised system that jointly learns verbal multiword expressions (VMWEs) and dependency parse trees as an auxiliary task. The model benefits from pre-trained multilingual BERT. BERT hidden layers are shared among the two tasks and we introduce an additional linear layer to retrieve VMWE tags. The dependency parse tree prediction is modelled by a linear layer and a bilinear one plus a tree CRF on top of BERT. The system has participated in the open track of the PARSEME shared task 2020 and ranked first in terms of F1-score in identifying unseen VMWEs as well as VMWEs in general, averaged across all 14 languages.

📄 PDF Abstract BibTeX arXiv:2011.02541

Code (1)

shivaat/MTLB-STRUCT 공식 구현 pytorch

Tasks

Multi-Task Learning

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

WordPiece 설명 없음

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

CRF Conditional Random Fields or CRFs are a type of probabilistic graph model that take neighboring sample context into account for tasks like classification. Prediction is…

Residual Connection 설명 없음

Multilingual Multiword Expression Identification Using Lateral Inhibition and Domain Adaptation

2023-06-17 · Andrei-Marius Avram, Verginica Barbu Mititelu, Vasile Păiş, Dumitru-Clementin Cercel 외

Correctly identifying multiword expressions (MWEs) is an important task for most natural language processing systems since their misidentification can result in ambiguity and misunderstanding of the underlying text. In t…

Domain Adaptation

ERMI at PARSEME Shared Task 2020: Embedding-Rich Multiword Expression Identification

2020-12-01 · COLING (MWE) 2020 12 · Zeynep Yirmibeşoğlu, Tunga Güngör

This paper describes the ERMI system submitted to the closed track of the PARSEME shared task 2020 on automatic identification of verbal multiword expressions (VMWEs). ERMI is an embedding-rich bidirectional LSTM-CRF mod…

POS TAG

Seen2Unseen at PARSEME Shared Task 2020: All Roads do not Lead to Unseen Verb-Noun VMWEs

2020-12-01 · COLING (MWE) 2020 12 · Caroline Pasquer, Agata Savary, Carlos Ramisch, Jean-Yves Antoine

We describe the Seen2Unseen system that participated in edition 1.2 of the PARSEME shared task on automatic identification of verbal multiword expressions (VMWEs). The identification of VMWEs that do not appear in the pr…

All Translation

Romanian Multiword Expression Detection Using Multilingual Adversarial Training and Lateral Inhibition

2023-04-22 · Andrei-Marius Avram, Verginica Barbu Mititelu, Dumitru-Clementin Cercel

Multiword expressions are a key ingredient for developing large-scale and linguistically sound natural language processing technology. This paper describes our improvements in automatically identifying Romanian multiword…

Edition 1.2 of the PARSEME Shared Task on Semi-supervised Identification of Verbal Multiword Expressions

2020-12-01 · COLING (MWE) 2020 12 · Carlos Ramisch, Agata Savary, Bruno Guillaume, Jakub Waszczuk 외

We present edition 1.2 of the PARSEME shared task on identification of verbal multiword expressions (VMWEs). Lessons learned from previous editions indicate that VMWEs have low ambiguity, and that the major challenge lie…