Towards Better UD Parsing: Deep Contextualized Word Embeddings, Ensemble, and Treebank Concatenation
This paper describes our system (HIT-SCIR) submitted to the CoNLL 2018 shared task on Multilingual Parsing from Raw Text to Universal Dependencies. We base our submission on Stanford's winning system for the CoNLL 2017 shared task and make two effective extensions: 1) incorporating deep contextualized word embeddings into both the part of speech tagger and parser; 2) ensembling parsers trained with different initialization. We also explore different ways of concatenating treebanks for further improvements. Experimental results on the development data show the effectiveness of our methods. In the final evaluation, our system was ranked first according to LAS (75.84%) and outperformed the other systems by a large margin.
Code (1)
Tasks
Dependency ParsingWord EmbeddingsSimilar Papers 제목 키워드 기반
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages
We use the multilingual OSCAR corpus, extracted from Common Crawl via language classification, filtering and cleaning, to train monolingual contextualized word embeddings (ELMo) for five mid-resource languages. We then c…
Part-Of-Speech TaggingWord EmbeddingsEvaluating Contextualized Embeddings on 54 Languages in POS Tagging, Lemmatization and Dependency Parsing
We present an extensive evaluation of three recently proposed methods for contextualized embeddings on 89 corpora in 54 languages of the Universal Dependencies 2.3 in three tasks: POS tagging, lemmatization, and dependen…
Dependency ParsingLemmatizationPOSPOS Tagging+1What Does This Word Mean? Explaining Contextualized Embeddings with Natural Language Definition
Contextualized word embeddings have boosted many NLP tasks compared with traditional static word embeddings. However, the word with a specific sense may have different contextualized embeddings due to its various context…
Word EmbeddingsCross-Lingual BERT Transformation for Zero-Shot Dependency Parsing
This paper investigates the problem of learning cross-lingual representations in a contextual space. We propose Cross-Lingual BERT Transformation (CLBT), a simple and efficient approach to generate cross-lingual contextu…
Cross-Lingual TransferDependency ParsingLanguage ModelingLanguage Modelling+2Improving Disfluency Detection by Self-Training a Self-Attentive Model
Self-attentive neural syntactic parsers using contextualized word embeddings (e.g. ELMo or BERT) currently produce state-of-the-art results in joint parsing and disfluency detection in speech transcripts. Since the conte…
Word Embeddings