Papers XLM-R
“XLM-R” 태그가 달린 논문 221편 · 필터 해제
TGIF: Tree-Graph Integrated-Format Parser for Enhanced UD with Two-Stage Generic- to Individual-Language Finetuning
We present our contribution to the IWPT 2021 shared task on parsing into enhanced Universal Dependencies. Our main system component is a hybrid tree-graph parser that integrates (a) predictions of spanning trees for the …
SentenceSentence segmentationXLM-RA Primer on Pretrained Multilingual Language Models
Multilingual Language Models (\MLLMs) such as mBERT, XLM, XLM-R, \textit{etc.} have emerged as a viable option for bringing the power of pretraining to a large number of languages. Given their success in zero-shot transf…
Joint Multilingual Sentence RepresentationsMultilingual text classificationMultilingual Word EmbeddingsPretrained Multilingual Language Models+3Automatic Sexism Detection with Multilingual Transformer Models
Sexism has become an increasingly major problem on social networks during the last years. The first shared task on sEXism Identification in Social neTworks (EXIST) at IberLEF 2021 is an international competition in the f…
Binary ClassificationClassificationTask 2Unsupervised Pre-training+1Investigating Transfer Learning in Multilingual Pre-trained Language Models through Chinese Natural Language Inference
Multilingual transformers (XLM, mT5) have been shown to have remarkable transfer skills in zero-shot settings. Most transfer studies, however, rely on automatically translated resources (XNLI, XQuAD), making it hard to d…
Cross-Lingual TransferNatural Language InferenceTransfer LearningXLM-RHow to Adapt Your Pretrained Multilingual Model to 1600 Languages
Pretrained multilingual models (PMMs) enable zero-shot learning via cross-lingual transfer, performing best for languages seen during pretraining. While methods exist to improve performance for unseen languages, they hav…
Cross-Lingual TransferNERPart-Of-Speech TaggingXLM-R+1Diagnosing Transformers in Task-Oriented Semantic Parsing
Modern task-oriented semantic parsing approaches typically use seq2seq transformers to map textual utterances to semantic frames comprised of intents and slots. While these models are empirically strong, their specific s…
Semantic ParsingvalidXLM-RXeroAlign: Zero-Shot Cross-lingual Transformer Alignment
The introduction of pretrained cross-lingual language models brought decisive improvements to multilingual NLP tasks. However, the lack of labelled task data necessitates a variety of methods aiming to close the gap to h…
Multilingual NLPNatural Language UnderstandingSentenceSentence Embeddings+3Larger-Scale Transformers for Multilingual Masked Language Modeling
Recent work has demonstrated the effectiveness of cross-lingual language model pretraining for cross-lingual understanding. In this study, we present the results of two larger multilingual masked language models, with 3.…
Language ModelingLanguage ModellingMasked Language ModelingXLM-RMultilingual and Zero-Shot is Closing in on Monolingual Web Register Classification
This article studies register classification of documents from the unrestricted web, such as news articles or opinion blogs, in a multilingual setting, exploring both the benefit of training on multiple languages and the…
ArticlesCross-Lingual TransferXLM-RZero-Shot Cross-Lingual TransferXLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond
Language models are ubiquitous in current NLP, and their multilingual capacity has recently attracted considerable attention. However, current analyses have almost exclusively focused on (multilingual variants of) standa…
Language ModellingSentiment AnalysisXLM-RX-METRA-ADA: Cross-lingual Meta-Transfer Learning Adaptation to Natural Language Understanding and Question Answering
Multilingual models, such as M-BERT and XLM-R, have gained increasing popularity, due to their zero-shot cross-lingual transfer learning capabilities. However, their generalization ability is still inconsistent for typol…
Cross-Lingual TransferMeta-LearningNatural Language UnderstandingQuestion Answering+3AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages
Pretrained multilingual models are able to perform cross-lingual transfer in a zero-shot setting, even for languages unseen during pretraining. However, prior work evaluating performance on unseen languages has largely b…
Cross-Lingual TransferNatural Language UnderstandingTranslationXLM-R+1Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Training
Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potential for zero-shot cross-lingual transfer. However, these multilingual encoders do not precisely align words and phrases ac…
Cross-Lingual TransferSentencetext-classificationText Classification+3Emotion Classification in a Resource Constrained Language Using Transformer-based Approach
Although research on emotion classification has significantly progressed in high-resource languages, it is still infancy for resource-constrained languages like Bengali. However, unavailability of necessary language proc…
ClassificationEmotion ClassificationGeneral ClassificationXLM-RBilingual alignment transfers to multilingual alignment for unsupervised parallel text mining
This work presents methods for learning cross-lingual sentence representations using paired or unpaired bilingual texts. We hypothesize that the cross-lingual alignment strategy is transferable, and therefore a model tra…
Language ModelingLanguage ModellingRetrievalSentence+1COVID-19 Named Entity Recognition for Vietnamese
The current COVID-19 pandemic has lead to the creation of many corpora that facilitate NLP research and downstream applications to help fight the pandemic. However, most of these corpora are exclusively for English. As t…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition In VietnameseNamed Entity Recognition (NER)+3MCL@IITK at SemEval-2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation using Augmented Data, Signals, and Transformers
In this work, we present our approach for solving the SemEval 2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation (MCL-WiC). The task is a sentence pair classification problem where the goal is to …
Cross-Lingual TransferSentenceSentence-Pair ClassificationTask 2+1Challenges in Annotating and Parsing Spoken, Code-switched, Frisian-Dutch Data
While high performance have been obtained for high-resource languages, performance on low-resource languages lags behind. In this paper we focus on the parsing of the low-resource language Frisian. We use a sample of cod…
XLM-RPriberam Labs at the 3rd Shared Task on SlavNER
This document describes our participation at the 3rd Shared Task on SlavNER, part of the 8th Balto-Slavic Natural Language Processing Workshop, where we focused exclusively in the Named Entity Recognition (NER) task. We …
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+1Benchmarking Pre-trained Language Models for Multilingual NER: TraSpaS at the BSNLP2021 Shared Task
In this paper we describe TraSpaS, a submission to the third shared task on named entity recognition hosted as part of the Balto-Slavic Natural Language Processing (BSNLP) Workshop. In it we evaluate various pre-trained …
BenchmarkingLanguage ModelingLanguage Modellingnamed-entity-recognition+4