Polyglot Contextual Representations Improve Crosslingual Transfer
We introduce Rosita, a method to produce multilingual contextual word representations by training a single language model on text from multiple languages. Our method combines the advantages of contextual word representations with those of multilingual representation learning. We produce language models from dissimilar language pairs (English/Arabic and English/Chinese) and use them in dependency parsing, semantic role labeling, and named entity recognition, with comparisons to monolingual and non-contextual variants. Our results provide further evidence for the benefits of polyglot learning, in which representations are shared across multiple languages.
Code (1)
Tasks
Dependency ParsingLanguage ModelingLanguage Modellingnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)Representation LearningSemantic Role LabelingSimilar Papers 제목 키워드 기반
Low-Resource Parsing with Crosslingual Contextualized Representations
Despite advances in dependency parsing, languages with small treebanks still present challenges. We assess recent approaches to multilingual contextual word representations (CWRs), and compare them for crosslingual trans…
Dependency ParsingCrosslingual Topic Modeling with WikiPDA
We present Wikipedia-based Polyglot Dirichlet Allocation (WikiPDA), a crosslingual topic model that learns to represent Wikipedia articles written in any language as distributions over a common set of language-independen…
ArticlesMatrix CompletionA Survey on Contextual Embeddings
Contextual embeddings, such as ELMo and BERT, move beyond global word representations like Word2Vec and achieve ground-breaking performance on a wide range of natural language processing tasks. Contextual embeddings assi…
Model CompressionSurveyCrosslingual Transfer Learning for Relation and Event Extraction via Word Category and Class Alignments
Previous work on crosslingual Relation and Event Extraction (REE) suffers from the monolingual bias issue due to the training of models on only the source language data. An approach to overcome this issue is to use unlab…
Event ExtractionRelationRepresentation LearningTransfer LearningMultilingual Information Extraction with PolyglotIE
We present PolyglotIE, a web-based tool for developing extractors that perform Information Extraction (IE) over multilingual data. Our tool has two core features: First, it allows users to develop extractors against a un…
Semantic Parsing