Papers XLM-R
“XLM-R” 태그가 달린 논문 221편 · 필터 해제
Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots
Multilingual models have demonstrated impressive cross-lingual transfer performance. However, test sets like XNLI are monolingual at the example level. In multilingual communities, it is common for polyglots to code-mix …
Cross-Lingual TransferXLM-RLightMBERT: A Simple Yet Effective Method for Multilingual BERT Distillation
The multilingual pre-trained language models (e.g, mBERT, XLM and XLM-R) have shown impressive performance on cross-lingual natural language understanding tasks. However, these models are computationally intensive and di…
Natural Language UnderstandingXLM-RAutomatic Difficulty Classification of Arabic Sentences
In this paper, we present a Modern Standard Arabic (MSA) Sentence difficulty classifier, which predicts the difficulty of sentences for language learners using either the CEFR proficiency levels or the binary classificat…
Binary ClassificationClassificationGeneral ClassificationPOS+5Vyākarana: A Colorless Green Benchmark for Syntactic Evaluation in Indic Languages
While there has been significant progress towards developing NLU resources for Indic languages, syntactic evaluation has been relatively less explored. Unlike English, Indic languages have rich morphosyntax, grammatical …
Depth EstimationDepth PredictionPOSPOS Tagging+2NLP-CUET@DravidianLangTech-EACL2021: Offensive Language Detection from Multilingual Code-Mixed Text using Transformers
The increasing accessibility of the internet facilitated social media usage and encouraged individuals to express their opinions liberally. Nevertheless, it also creates a place for content polluters to disseminate offen…
Multilingual text classificationXLM-RBootstrapping Multilingual AMR with Contextual Word Alignments
We develop high performance multilingualAbstract Meaning Representation (AMR) sys-tems by projecting English AMR annotationsto other languages with weak supervision. Weachieve this goal by bootstrapping transformer-based…
Multilingual Word EmbeddingsWord AlignmentWord EmbeddingsXLM-RLOME: Large Ontology Multilingual Extraction
We present LOME, a system for performing multilingual information extraction. Given a text document as input, our core system identifies spans of textual entity and event mentions with a FrameNet (Baker et al., 1998) par…
coreference-resolutionCoreference ResolutionEntity TypingRelation+3Distilling Large Language Models into Tiny and Effective Students using pQRNN
Large pre-trained multilingual models like mBERT, XLM-R achieve state of the art results on language understanding tasks. However, they are not well suited for latency critical applications on both servers and edge devic…
Data AugmentationSemantic ParsingXLM-RMiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
We generalize deep self-attention distillation in MiniLM (Wang et al., 2020) by only using self-attention relation distillation for task-agnostic compression of pretrained Transformers. In particular, we define multi-hea…
RelationXLM-RARBERT & MARBERT: Deep Bidirectional Transformers for Arabic
Pre-trained language models (LMs) are currently integral to many natural language processing systems. Although multilingual LMs were also introduced to serve many languages, these have limitations such as being costly at…
DiversityXLM-RCogALex-VI Shared Task: Transrelation - A Robust Multilingual Language Model for Multilingual Relation Identification
We describe our submission to the CogALex-VI shared task on the identification of multilingual paradigmatic relations building on XLM-RoBERTa (XLM-R), a robustly optimized and multilingual BERT model. In spite of several…
Hypernym DiscoveryLanguage ModelingLanguage ModellingMultilingual text classification+3SenseCluster at SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection
We (Team Skurt) propose a simple method to detect lexical semantic change by clustering contextualized embeddings produced by XLM-R, using K-Means++. The basic idea is that contextualized embeddings that encode the same …
Change DetectionClusteringXLM-RCoLi at UdS at SemEval-2020 Task 12: Offensive Tweet Detection with Ensembling
We present our submission and results for SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) where we participated in offensive tweet classification tasks in English, A…
BIG-bench Machine LearningLanguage IdentificationregressionXLM-RTeam Rouges at SemEval-2020 Task 12: Cross-lingual Inductive Transfer to Detect Offensive Language
With the growing use of social media and its availability, many instances of the use of offensive language have been observed across multiple languages and domains. This phenomenon has given rise to the growing need to d…
Language IdentificationPositionXLM-RZero-Shot LearningDetecting Urgency Status of Crisis Tweets: A Transfer Learning Approach for Low Resource Languages
We release an urgency dataset that consists of English tweets relating to natural crises, along with annotations of their corresponding urgency status. Additionally, we release evaluation datasets for two low-resource la…
Transfer LearningXLM-RA Multilingual Reading Comprehension System for more than 100 Languages
This paper presents M-GAAMA, a Multilingual Question Answering architecture and demo system. This is the first multilingual machine reading comprehension (MRC) demo which is able to answer questions in over 100 languages…
Machine Reading ComprehensionMachine TranslationQuestion AnsweringReading Comprehension+2From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual Transformers
Massively multilingual transformers (MMTs) pretrained via language modeling (e.g., mBERT, XLM-R) have become a default paradigm for zero-shot language transfer in NLP, offering unmatched transfer performance. Current eva…
Cross-Lingual Word EmbeddingsDependency ParsingLanguage ModelingLanguage Modelling+5XGLUE: A New Benchmark Datasetfor Cross-lingual Pre-training, Understanding and Generation
In this paper, we introduce XGLUE, a new benchmark dataset to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora, and evaluate their performance across a diverse set of cross-ling…
Natural Language UnderstandingXLM-RCross-neutralising: Probing for joint encoding of linguistic information in multilingual models
Multilingual sentence encoders are widely used to transfer NLP models across languages. The success of this transfer is, however, dependent on the model's ability to encode the patterns of cross-lingual similarity and va…
SentenceXLM-RApplying Occam's Razor to Transformer-Based Dependency Parsing: What Works, What Doesn't, and What is Really Necessary
The introduction of pre-trained transformer-based contextualized word embeddings has led to considerable improvements in the accuracy of graph-based parsers for frameworks such as Universal Dependencies (UD). However, pr…
Dependency ParsingPart-Of-Speech TaggingWord EmbeddingsXLM-R