Evaluating Contextualized Embeddings on 54 Languages in POS Tagging, Lemmatization and Dependency Parsing

2019-08-20 · Milan Straka, Jana Straková, Jan Hajič

We present an extensive evaluation of three recently proposed methods for contextualized embeddings on 89 corpora in 54 languages of the Universal Dependencies 2.3 in three tasks: POS tagging, lemmatization, and dependency parsing. Employing the BERT, Flair and ELMo as pretrained embedding inputs in a strong baseline of UDPipe 2.0, one of the best-performing systems of the CoNLL 2018 Shared Task and an overall winner of the EPE 2018, we present a one-to-one comparison of the three contextualized word embedding methods, as well as a comparison with word2vec-like pretrained embeddings and with end-to-end character-level word embeddings. We report state-of-the-art results in all three tasks as compared to results on UD 2.2 in the CoNLL 2018 Shared Task.

📄 PDF Abstract BibTeX arXiv:1908.07448

Code (0)

등록된 구현이 없습니다.

Tasks

Dependency ParsingLemmatizationPOSPOS TaggingWord Embeddings

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

Weight Decay 설명 없음

Residual Connection 설명 없음

Adam 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Attention 설명 없음

Czech Text Processing with Contextual Embeddings: POS Tagging, Lemmatization, Parsing and NER

2019-09-08 · Milan Straka, Jana Straková, Jan Hajič

Contextualized embeddings, which capture appropriate word meaning depending on context, have recently been proposed. We evaluate two meth ods for precomputing such embeddings, BERT and Flair, on four Czech text processin…

Dependency Parsing Lemmatization named-entity-recognition Named Entity Recognition+5

UDPipe at EvaLatin 2020: Contextualized Embeddings and Treebank Embeddings

2020-06-05 · LREC 2020 5 · Milan Straka, Jana Straková

We present our contribution to the EvaLatin shared task, which is the first evaluation campaign devoted to the evaluation of NLP tools for Latin. We submitted a system based on UDPipe 2.0, one of the winners of the CoNLL…

Lemmatization POS POS Tagging

LemMED: Fast and Effective Neural Morphological Analysis with Short Context Windows

2020-10-21 · Aibek Makazhanov, Sharon Goldwater, Adam Lopez

We present LemMED, a character-level encoder-decoder for contextual morphological analysis (combined lemmatization and tagging). LemMED extends and is named after two other attention-based models, namely Lematus, a conte…

Decoder Lemmatization Morphological Analysis Sentence

A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages

2020-06-11 · ACL 2020 6 · Pedro Javier Ortiz Suárez, Laurent Romary, Benoît Sagot

We use the multilingual OSCAR corpus, extracted from Common Crawl via language classification, filtering and cleaning, to train monolingual contextualized word embeddings (ELMo) for five mid-resource languages. We then c…

Part-Of-Speech Tagging Word Embeddings

Under-resourced studies of under-resourced languages: lemmatization and POS-tagging with LLM annotators for historical Armenian, Georgian, Greek and Syriac

2026-02-17 · Chahan Vidal-Gorène, Bastien Kindt, Florian Cafiero arxiv

Low-resource languages pose persistent challenges for Natural Language Processing tasks such as lemmatization and part-of-speech (POS) tagging. This paper investigates the capacity of recent large language models (LLMs),…