Papers XLM-R
“XLM-R” 태그가 달린 논문 221편 · 필터 해제
A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning
Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries, which are expensive and impractical for low-resource languages. To disengage from these dependencies, researchers have…
Cross-Lingual Transfertext-classificationText ClassificationTransfer Learning+2Alibaba-Translate China's Submission for WMT 2022 Quality Estimation Shared Task
In this paper, we present our submission to the sentence-level MQM benchmark at Quality Estimation Shared Task, named UniTE (Unified Translation Evaluation). Specifically, our systems employ the framework of UniTE, which…
Language ModelingLanguage ModellingSentenceXLM-RAre Pretrained Multilingual Models Equally Fair Across Languages?
Pretrained multilingual language models can help bridge the digital language divide, enabling high-quality NLP models for lower resourced languages. Studies of multilingual models have so far focused on performance, cons…
Cloze TestFairnessPretrained Multilingual Language ModelsXLM-RHow about Time? Probing a Multilingual Language Model for Temporal Relations
This paper presents a comprehensive set of probing experiments using a multilingual language model, XLM-R, for temporal relation classification between events in four languages. Results show an advantage of contextualize…
Language ModelingLanguage ModellingRelationRelation Classification+3Disfluency Detection for Vietnamese
In this paper, we present the first empirical study for Vietnamese disfluency detection. To conduct this study, we first create a disfluency detection dataset for Vietnamese, with manual annotations over two disfluency t…
Vietnamese Word SegmentationXLM-RExtending Word-Level Quality Estimation for Post-Editing Assistance
We define a novel concept called extended word alignment in order to improve post-editing assistance efficiency. Based on extended word alignment, we further propose a novel task called refined word-level QE that outputs…
Word AlignmentXLM-RSMTCE: A Social Media Text Classification Evaluation Benchmark and BERTology Models for Vietnamese
Text classification is a typical natural language processing or computational linguistics task with various interesting applications. As the number of users on social media platforms increases, data acceleration promotes…
Classificationtext-classificationText ClassificationXLM-RALEXSIS-PT: A New Resource for Portuguese Lexical Simplification
Lexical simplification (LS) is the task of automatically replacing complex words for easier ones making texts more accessible to various target populations (e.g. individuals with low literacy, individuals with learning d…
ArticlesLexical SimplificationXLM-RFrom Disfluency Detection to Intent Detection and Slot Filling
We present the first empirical study investigating the influence of disfluency detection on downstream tasks of intent detection and slot filling. We perform this study for Vietnamese -- a low-resource language that has …
Intent DetectionLanguage ModelingLanguage Modellingslot-filling+2Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matching
Previous studies have proved that cross-lingual knowledge distillation can significantly improve the performance of pre-trained models for cross-lingual similarity matching tasks. However, the student model needs to be l…
Contrastive LearningKnowledge DistillationSemantic SimilaritySemantic Textual Similarity+1Multilingual Bidirectional Unsupervised Translation Through Multilingual Finetuning and Back-Translation
We propose a two-stage approach for training a single NMT model to translate unseen languages both to and from English. For the first stage, we initialize an encoder-decoder model to pretrained XLM-R and RoBERTa weights,…
DecoderNMTTranslationXLM-RNeural Approaches to Multilingual Information Retrieval
Providing access to information across languages has been a goal of Information Retrieval (IR) for decades. While progress has been made on Cross Language IR (CLIR) where queries are expressed in one language and documen…
Document TranslationInformation RetrievalLanguage ModelingLanguage Modelling+3Investigating Language Relationships in Multilingual Sentence Encoders Through the Lens of Linguistic Typology
Multilingual sentence encoders have seen much success in cross-lingual model transfer for downstream NLP tasks. The success of this transfer is, however, dependent on the model’s ability to encode the patterns of cross-l…
SentenceXLM-RBERTifying Sinhala -- A Comprehensive Analysis of Pre-trained Language Models for Sinhala Text Classification
This research provides the first comprehensive analysis of the performance of pre-trained language models for Sinhala text classification. We test on a set of different Sinhala text classification tasks and our analysis …
Classificationtext-classificationText ClassificationXLM-RMassively Multilingual Lexical Specialization of Multilingual Transformers
While pretrained language models (PLMs) primarily serve as general-purpose text encoders that can be fine-tuned for a wide variety of downstream tasks, recent work has shown that they can also be rewired to produce high-…
Bilingual Lexicon InductionRetrievalSentenceSentence Retrieval+3AsNER -- Annotated Dataset and Baseline for Assamese Named Entity recognition
We present the AsNER, a named entity annotation dataset for low resource Assamese language with a baseline Assamese NER model. The dataset contains about 99k tokens comprised of text from the speech of the Prime Minister…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+1Hitachi at SemEval-2022 Task 2: On the Effectiveness of Span-based Classification Approaches for Multilingual Idiomaticity Detection
In this paper, we describe our system for SemEval-2022 Task 2: Multilingual Idiomaticity Detection and Sentence Embedding. The task aims at detecting idiomaticity in an input sequence (Subtask A) and modeling representat…
ClassificationSentenceSentence EmbeddingSentence-Embedding+2Sliced at SemEval-2022 Task 11: Bigger, Better? Massively Multilingual LMs for Multilingual Complex NER on an Academic GPU Budget
Massively multilingual language models (MMLMs) have become a widely-used representation method, and multiple large MMLMs were proposed in recent years. A trend is to train MMLMs on larger text corpora or with more layers…
GPUNERXLM-RImproving Code-Switching Dependency Parsing with Semi-Supervised Auxiliary Tasks
Code-switching dependency parsing stands as a challenging task due to both the scarcity of necessary resources and the structural difficulties embedded in code-switched languages. In this study, we introduce novel sequen…
Dependency ParsingXLM-RAlexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems
We present results from a large-scale experiment on pretraining encoders with non-embedding parameter counts ranging from 700M to 9.3B, their subsequent distillation into smaller models ranging from 17M-170M parameters, …
Cross-Lingual Natural Language Inferenceintent-classificationIntent ClassificationNatural Language Inference+4