Papers XLM-R
“XLM-R” 태그가 달린 논문 221편 · 필터 해제
Galileo at SemEval-2020 Task 12: Multi-lingual Learning for Offensive Language Identification using Pre-trained Language Models
This paper describes Galileo's performance in SemEval-2020 Task 12 on detecting and categorizing offensive language in social media. For Offensive Language Identification, we proposed a multi-lingual method using Pre-tra…
AllKnowledge DistillationLanguage IdentificationXLM-RA Pilot Study of Text-to-SQL Semantic Parsing for Vietnamese
Semantic parsing is an important NLP task. However, Vietnamese is a low-resource language in this research area. In this paper, we present the first public large-scale Text-to-SQL semantic parsing dataset for Vietnamese.…
Language ModelingLanguage ModellingSemantic ParsingText to SQL+3What does it mean to be language-agnostic? Probing multilingual sentence encoders for typological properties
Multilingual sentence encoders have seen much success in cross-lingual model transfer for downstream NLP tasks. Yet, we know relatively little about the properties of individual languages or the general patterns of lingu…
SentenceXLM-RCross-lingual Transfer Learning for Semantic Role Labeling in Russian
This work is devoted to semantic role labeling (SRL) task in Russian. We investigate the role of transfer learning strategies between English FrameNet and Russian FrameBank corpora. We perform experiments with embeddings…
Cross-Lingual TransferSemantic Role LabelingTransfer LearningXLM-RGREEK-BERT: The Greeks visiting Sesame Street
Transformer-based language models, such as BERT and its variants, have achieved state-of-the-art performance in several downstream natural language processing (NLP) tasks on generic benchmark datasets (e.g., GLUE, SQUAD,…
Language ModelingLanguage Modellingnamed-entity-recognitionNamed Entity Recognition+5Inducing Language-Agnostic Multilingual Representations
Cross-lingual representations have the potential to make NLP techniques available to the vast majority of languages in the world. However, they currently require large pretraining corpora or access to typologically simil…
Cross-Lingual TransferSentenceText NormalizationXLM-R+1On Learning Universal Representations Across Languages
Recent studies have demonstrated the overwhelming advantage of cross-lingual pre-trained models (PTMs), such as multilingual BERT and XLM, on cross-lingual NLP tasks. However, existing approaches essentially capture the …
Contrastive LearningCross-Lingual Natural Language InferenceLanguage ModelingLanguage Modelling+5AdapterHub: A Framework for Adapting Transformers
The current modus operandi in NLP involves downloading and fine-tuning pre-trained models consisting of millions or billions of parameters. Storing and sharing such large trained models is expensive, slow, and time-consu…
XLM-RCross-lingual Inductive Transfer to Detect Offensive Language
With the growing use of social media and its availability, many instances of the use of offensive language have been observed across multiple languages and domains. This phenomenon has given rise to the growing need to d…
Language IdentificationPositionXLM-RZero-Shot LearningA Bayesian Multilingual Document Model for Zero-shot Topic Identification and Discovery
In this paper, we present a Bayesian multilingual document model for learning language-independent document embeddings. The model is an extension of BaySMM [Kesiraju et al 2020] to the multilingual scenario. It learns to…
SentenceXLM-RFinEst BERT and CroSloEngual BERT: less is more in multilingual models
Large pretrained masked language models have become state-of-the-art solutions for many NLP problems. The research has been mostly focused on English language, though. While massively multilingual models exist, studies h…
Dependency ParsingNERPOSPOS Tagging+1English Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Too
Intermediate-task training has been shown to substantially improve pretrained model performance on many language understanding tasks, at least in monolingual English settings. Here, we investigate whether English interme…
Cross-Lingual TransferQuestion AnsweringXLM-RZero-Shot Cross-Lingual TransferEnglish Intermediate-Task Training Improves Zero-Shot Cross-Lingual Transfer Too
Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuning again on the target task---often improves model performance substantially on language understanding tasks in monoling…
Cross-Lingual TransferHellaSwagQuestion AnsweringRetrieval+4BERTweet: A pre-trained language model for English Tweets
We present BERTweet, the first public large-scale pre-trained language model for English Tweets. Our BERTweet, having the same architecture as BERT-base (Devlin et al., 2019), is trained using the RoBERTa pre-training pr…
Language ModelingLanguage Modellingnamed-entity-recognitionNamed Entity Recognition+6From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformers
Massively multilingual transformers pretrained with language modeling objectives (e.g., mBERT, XLM-R) have become a de facto default transfer paradigm for zero-shot cross-lingual transfer in NLP, offering unmatched trans…
Cross-Lingual TransferCross-Lingual Word EmbeddingsDependency ParsingLanguage Modeling+7Don't Use English Dev: On the Zero-Shot Cross-Lingual Evaluation of Contextual Embeddings
Multilingual contextual embeddings have demonstrated state-of-the-art performance in zero-shot cross-lingual transfer learning, where multilingual BERT is fine-tuned on one source language and evaluated on a different ta…
Cross-Lingual TransferModel SelectionTransfer LearningXLM-R+1MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer
The main goal behind state-of-the-art pre-trained multilingual models such as multilingual BERT and XLM-R is enabling and bootstrapping NLP applications in low-resource languages through zero-shot or few-shot cross-lingu…
Cross-Lingual Transfernamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+2XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
In this paper, we introduce XGLUE, a new benchmark dataset that can be used to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora and evaluate their performance across a diverse s…
Natural Language UnderstandingXLM-RTesting pre-trained Transformer models for Lithuanian news clustering
A recent introduction of Transformer deep learning architecture made breakthroughs in various natural language processing tasks. However, non-English languages could not leverage such new opportunities with the English t…
ClusteringXLM-RPhoBERT: Pre-trained language models for Vietnamese
We present PhoBERT with two versions, PhoBERT-base and PhoBERT-large, the first public large-scale monolingual language models pre-trained for Vietnamese. Experimental results show that PhoBERT consistently outperforms t…
Dependency Parsingnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+4