Papers XLM-R
“XLM-R” 태그가 달린 논문 221편 · 필터 해제
Targeted Multilingual Adaptation for Low-resource Language Families
The "massively-multilingual" training of multilingual models is known to limit their utility in any one language, and they perform particularly poorly on low-resource languages. However, there is evidence that low-resour…
XLM-RZero-Shot Tokenizer Transfer
Language models (LMs) are bound to their tokenizer, which maps raw text to a sequence of vocabulary items (tokens). This restricts their flexibility: for example, LMs trained primarily on English may still perform well i…
XLM-RSoftware Mention Recognition with a Three-Stage Framework Based on BERTology Models at SOMD 2024
This paper describes our systems for the sub-task I in the Software Mention Detection in Scholarly Publications shared-task. We propose three approaches leveraging different pre-trained language models (BERT, SciBERT, an…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+3Adapting Mental Health Prediction Tasks for Cross-lingual Learning via Meta-Training and In-context Learning with Large Language Model
Timely identification is essential for the efficient handling of mental health illnesses such as depression. However, the current research fails to adequately address the prediction of mental health conditions from socia…
Cross-Lingual TransferIn-Context LearningLanguage ModelingLanguage Modelling+3MaiNLP at SemEval-2024 Task 1: Analyzing Source Language Selection in Cross-Lingual Textual Relatedness
This paper presents our system developed for the SemEval-2024 Task 1: Semantic Textual Relatedness (STR), on Track C: Cross-lingual. The task aims to detect semantic relatedness of two sentences in a given target languag…
Cross-Lingual TransferData AugmentationMachine TranslationXLM-R+1Cross-Lingual Transfer Robustness to Lower-Resource Languages on Adversarial Datasets
Multilingual Language Models (MLLMs) exhibit robust cross-lingual transfer capabilities, or the ability to leverage information acquired in a source language and apply it to a target language. These capabilities find pra…
Cross-Lingual Transfernamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+2Solution for Emotion Prediction Competition of Workshop on Emotionally and Culturally Intelligent AI
This report provide a detailed description of the method that we explored and proposed in the WECIA Emotion Prediction Competition (EPC), which predicts a person's emotion through an artistic work with a comment. The dat…
DiversityXLM-RCICLe: Conformal In-Context Learning for Largescale Multi-Class Food Risk Classification
Contaminated or adulterated food poses a substantial risk to human health. Given sets of labeled web texts for training, Machine Learning and Natural Language Processing can be applied to automatically detect such risks.…
Conformal PredictionIn-Context LearningXLM-RMachines Do See Color: A Guideline to Classify Different Forms of Racist Discourse in Large Corpora
Current methods to identify and classify racist language in text rely on small-n qualitative approaches or large-n approaches focusing exclusively on overt forms of racist discourse. This article provides a step-by-step …
text-classificationText ClassificationXLM-RLinguAlchemy: Fusing Typological and Geographical Elements for Unseen Language Generalization
Pretrained language models (PLMs) have become remarkably adept at task and language generalization. Nonetheless, they often fail when faced with unseen languages. In this work, we present LinguAlchemy, a regularization m…
intent-classificationIntent ClassificationLanguage ModellingNews Classification+1Hate Speech and Offensive Content Detection in Indo-Aryan Languages: A Battle of LSTM and Transformers
Social media platforms serve as accessible outlets for individuals to express their thoughts and experiences, resulting in an influx of user-generated data spanning all age groups. While these platforms enable free expre…
Hate Speech DetectionModel SelectionXLM-RA Text-to-Text Model for Multilingual Offensive Language Identification
The ubiquity of offensive content on social media is a growing cause for concern among companies and government organizations. Recently, transformer-based models such as BERT, XLNET, and XLM-R have achieved state-of-the-…
DecoderLanguage IdentificationXLM-RKBioXLM: A Knowledge-anchored Biomedical Multilingual Pretrained Language Model
Most biomedical pretrained language models are monolingual and cannot handle the growing cross-lingual requirements. The scarcity of non-English domain corpora, not to mention parallel data, poses a significant hurdle in…
Language ModelingLanguage ModellingRelationRelation Prediction+1MELA: Multilingual Evaluation of Linguistic Acceptability
In this work, we present the largest benchmark to date on linguistic acceptability: Multilingual Evaluation of Linguistic Acceptability -- MELA, with 46K samples covering 10 languages from a diverse set of language famil…
Code GenerationCross-Lingual TransferLinguistic AcceptabilityMulti-Task Learning+1Zero-Shot Cross-Lingual Sentiment Classification under Distribution Shift: an Exploratory Study
The brittleness of finetuned language model performance on out-of-distribution (OOD) test samples in unseen domains has been well-studied for English, yet is unexplored for multi-lingual models. Therefore, we study gener…
Cross-Lingual Sentiment ClassificationCross-Lingual TransferLanguage ModellingSentiment Analysis+3Counterfactually Probing Language Identity in Multilingual Models
Techniques in causal analysis of language models illuminate how linguistic information is organized in LLMs. We use one such technique, AlterRep, a method of counterfactual probing, to explore the internal structure of m…
counterfactualLanguage ModelingLanguage ModellingMasked Language Modeling+1Lost in Translation, Found in Spans: Identifying Claims in Multilingual Social Media
Claim span identification (CSI) is an important step in fact-checking pipelines, aiming to identify text segments that contain a checkworthy claim or assertion in a social media post. Despite its importance to journalist…
Cross-Lingual TransferFact CheckingXLM-RImproving Cross-Lingual Transfer through Subtree-Aware Word Reordering
Despite the impressive growth of the abilities of multilingual language models, such as XLM-R and mT5, it has been shown that they still face difficulties when tackling typologically-distant languages, particularly in th…
Cross-Lingual TransferPOSXLM-RMedAI Dialog Corpus (MEDIC): Zero-Shot Classification of Doctor and AI Responses in Health Consultations
Zero-shot classification enables text to be classified into classes not seen during training. In this study, we examine the efficacy of zero-shot learning models in classifying healthcare consultation responses from Doct…
Classificationtext-classificationText ClassificationXLM-R+2ViSoBERT: A Pre-Trained Language Model for Vietnamese Social Media Text Processing
English and Chinese, known as resource-rich languages, have witnessed the strong development of transformer-based language models for natural language processing tasks. Although Vietnam has approximately 100M people spea…
Language ModelingLanguage ModellingVietnamese Hate Speech DetectionVietnamese Language Models+2