Papers XLM-R
“XLM-R” 태그가 달린 논문 221편 · 필터 해제
Debating Europe: A Multilingual Multi-Target Stance Classification Dataset of Online Debates
We present a new dataset of online debates in English, annotated with stance. The dataset was scraped from the “Debating Europe” platform, where users exchange opinions over different subjects related to the European Uni…
Stance ClassificationXLM-RAsNER - Annotated Dataset and Baseline for Assamese Named Entity recognition
We present the AsNER, a named entity annotation dataset for low resource Assamese language with a baseline Assamese NER model. The dataset contains about 99k tokens comprised of text from the speech of the Prime Minister…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+1BERTifying Sinhala - A Comprehensive Analysis of Pre-trained Language Models for Sinhala Text Classification
This research provides the first comprehensive analysis of the performance of pre-trained language models for Sinhala text classification. We test on a set of different Sinhala text classification tasks and our analysis …
Classificationtext-classificationText ClassificationXLM-RVLUE: A Multi-Task Benchmark for Evaluating Vision-Language Models
Recent advances in vision-language pre-training (VLP) have demonstrated impressive performance in a range of vision-language (VL) tasks. However, there exist several challenges for measuring the community's progress in b…
Vietnamese Natural Language UnderstandingXLM-RAnalyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models
The emergent cross-lingual transfer seen in multilingual pretrained models has sparked significant interest in studying their behavior. However, because these analyses have focused on fully trained multilingual models, l…
Cross-Lingual TransferXLM-RThe Geometry of Multilingual Language Model Representations
We assess how multilingual language models maintain a shared multilingual representation space while still encoding language-sensitive information in each language. Using XLM-R as a case study, we show that languages occ…
Cross-Lingual TransferLanguage ModelingLanguage Modellingmodel+2Multi-Task Text Classification using Graph Convolutional Networks for Large-Scale Low Resource Language
Graph Convolutional Networks (GCN) have achieved state-of-art results on single text classification tasks like sentiment analysis, emotion detection, etc. However, the performance is achieved by testing and reporting on …
Graph ReconstructionSarcasm DetectionSentenceSentiment Analysis+4Explaining Classes through Stable Word Attributions
Input saliency methods have recently become a popular tool for explaining predictions of deep learning models in NLP. Nevertheless, there has been little work investigating methods for aggregating prediction-level explan…
text-classificationText ClassificationXLM-Rblack[LSCDiscovery shared task] GlossReader at LSCDiscovery: Train to Select a Proper Gloss in English – Discover Lexical Semantic Change in Spanish
The contextualized embeddings obtained from neural networks pre-trained as Language Models (LM) or Masked Language Models (MLM) are not well suitable for solving the Lexical Semantic Change Detection (LSCD) task because …
Change DetectionXLM-RTowards Making the Most of Cross-Lingual Transfer for Zero-Shot Neural Machine Translation
This paper demonstrates that multilingual pretraining and multilingual fine-tuning are both critical for facilitating cross-lingual transfer in zero-shot translation, where the neural machine translation (NMT) model is t…
Abstractive Text SummarizationCross-Lingual Abstractive SummarizationCross-Lingual TransferDecoder+4TeluguNER: Leveraging Multi-Domain Named Entity Recognition with Deep Transformers
Named Entity Recognition (NER) is a successful and well-researched problem in English due to the availability of resources. The transformer models, specifically the masked-language models (MLM), have shown remarkable per…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+1CUET-NLP@TamilNLP-ACL2022: Multi-Class Textual Emotion Detection from Social Media using Transformer
Recently, emotion analysis has gained increased attention by NLP researchers due to its various applications in opinion mining, e-commerce, comprehensive search, healthcare, personalized recommendations and online educat…
Emotion RecognitionOpinion MiningXLM-RMASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages
We present the MASSIVE dataset--Multilingual Amazon Slu resource package (SLURP) for Slot-filling, Intent classification, and Virtual assistant Evaluation. MASSIVE contains 1M realistic, parallel, labeled virtual assista…
intent-classificationIntent ClassificationNatural Language UnderstandingSlot Filling+3Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning
Multilingual pre-trained language models (PLMs) have demonstrated impressive performance on several downstream tasks for both high-resourced and low-resourced languages. However, there is still a large performance drop f…
Cross-Lingual TransferLanguage ModellingNERparameter-efficient fine-tuning+6The Impact of Cross-Lingual Adjustment of Contextual Word Representations on Zero-Shot Transfer
Large multilingual language models such as mBERT or XLM-R enable zero-shot cross-lingual transfer in various IR and NLP tasks. Cao et al. (2020) proposed a data- and compute-efficient method for cross-lingual adjustment …
Continual LearningCross-Lingual TransferMachine Reading ComprehensionNatural Language Inference+3Do Not Fire the Linguist: Grammatical Profiles Help Language Models Detect Semantic Change
Morphological and syntactic changes in word usage (as captured, e.g., by grammatical profiles) have been shown to be good predictors of a word's meaning change. In this work, we explore whether large pre-trained contextu…
Change DetectionLanguage ModelingLanguage ModellingXLM-RFactual Consistency of Multilingual Pretrained Language Models
Pretrained language models can be queried for factual knowledge, with potential applications in knowledge base acquisition and tasks that require inference. However, for that, we need to know how reliable this knowledge …
XLM-RTowards Explainable Evaluation Metrics for Natural Language Generation
Unlike classical lexical overlap metrics such as BLEU, most current evaluation metrics (such as BERTScore or MoverScore) are based on black-box language models such as BERT or XLM-R. They often achieve strong correlation…
Machine TranslationText GenerationTranslationXLM-RDo Multilingual Language Models Capture Differing Moral Norms?
Massively multilingual sentence representations are trained on large corpora of uncurated data, with a very imbalanced proportion of languages included in the training. This may cause the models to grasp cultural values …
SentenceXLM-RCombining Static and Contextualised Multilingual Embeddings
Static and contextual multilingual embeddings have complementary strengths. Static embeddings, while less expressive than contextual language models, can be more straightforwardly aligned across multiple languages. We co…
RetrievalXLM-R