Papers XLM-R
“XLM-R” 태그가 달린 논문 221편 · 필터 해제
Cross-Linguistic Transfer in Multilingual NLP: The Role of Language Families and Morphology
Cross-lingual transfer has become a crucial aspect of multilingual NLP, as it allows for models trained on resource-rich languages to be applied to low-resource languages more effectively. Recently massively multilingual…
Cross-Lingual TransferMultilingual NLPXLM-RSubasa -- Adapting Language Models for Low-resourced Offensive Language Detection in Sinhala
Accurate detection of offensive language is essential for a number of applications related to social media safety. There is a sharp contrast in performance in this task between low and high-resource languages. In this pa…
XLM-RMultilingual Encoder Knows more than You Realize: Shared Weights Pretraining for Extremely Low-Resource Languages
While multilingual language models like XLM-R have advanced multilingualism in NLP, they still perform poorly in extremely low-resource languages. This situation is exacerbated by the fact that modern LLMs such as LLaMA …
DecoderText GenerationXLM-RAmaSQuAD: A Benchmark for Amharic Extractive Question Answering
This research presents a novel framework for translating extractive question-answering datasets into low-resource languages, as demonstrated by the creation of the AmaSQuAD dataset, a translation of SQuAD 2.0 into Amhari…
Extractive Question-AnsweringQuestion AnsweringXLM-REvaluating the Effectiveness of XAI Techniques for Encoder-Based Language Models
The black-box nature of large language models (LLMs) necessitates the development of eXplainable AI (XAI) techniques for transparency and trustworthiness. However, evaluating these techniques remains a challenge. This st…
XLM-RFuocChuVIP123 at CoMeDi Shared Task: Disagreement Ranking with XLM-Roberta Sentence Embeddings and Deep Neural Regression
This paper presents results of our system for CoMeDi Shared Task, focusing on Subtask 2: Disagreement Ranking. Our system leverages sentence embeddings generated by the paraphrase-xlm-r-multilingual-v1 model, combined wi…
SentenceSentence EmbeddingsXLM-RComparative Approaches to Sentiment Analysis Using Datasets in Major European and Arabic Languages
This study explores transformer-based models such as BERT, mBERT, and XLM-R for multi-lingual sentiment analysis across diverse linguistic structures. Key contributions include the identification of XLM-R superior adapta…
Sentiment AnalysisSentiment ClassificationXLM-RMulti-stage Training of Bilingual Islamic LLM for Neural Passage Retrieval
This study examines the use of Natural Language Processing (NLP) technology within the Islamic domain, focusing on developing an Islamic neural retrieval model. By leveraging the robust XLM-R model, the research employs …
Data AugmentationDomain AdaptationLanguage ModelingLanguage Modelling+4BabyLMs for isiXhosa: Data-Efficient Language Modelling in a Low-Resource Context
The BabyLM challenge called on participants to develop sample-efficient language models. Submissions were pretrained on a fixed English corpus, limited to the amount of words children are exposed to in development (<100m…
Language ModellingNERPOSPOS Tagging+1USTCCTSU at SemEval-2024 Task 1: Reducing Anisotropy for Cross-lingual Semantic Textual Relatedness Task
Cross-lingual semantic textual relatedness task is an important research task that addresses challenges in cross-lingual communication and text understanding. It helps establish semantic connections between different lan…
Information RetrievalMachine TranslationRetrievalSentence+1Retrofitting Large Language Models with Dynamic Tokenization
Current language models (LMs) use a fixed, static subword tokenizer. This default choice typically results in degraded efficiency and language capabilities, especially in languages other than English. To address this iss…
DecoderFairnessXLM-RTransformer-Based Contextualized Language Models Joint with Neural Networks for Natural Language Inference in Vietnamese
Natural Language Inference (NLI) is a task within Natural Language Processing (NLP) that holds value for various AI applications. However, there have been limited studies on Natural Language Inference in Vietnamese that …
Natural Language InferenceXLM-RFrom N-grams to Pre-trained Multilingual Models For Language Identification
In this paper, we investigate the use of N-gram models and Large Pre-trained Multilingual models for Language Identification (LID) across 11 South African languages. For N-gram models, this study shows that effective dat…
Language IdentificationXLM-RLangSAMP: Language-Script Aware Multilingual Pretraining
Recent multilingual pretrained language models (mPLMs) often avoid using language embeddings -- learnable vectors assigned to different languages. These embeddings are discarded for two main reasons: (1) mPLMs are expect…
Continual PretrainingLanguage ModelingLanguage ModellingRepresentation Learning+1GrEmLIn: A Repository of Green Baseline Embeddings for 87 Low-Resource Languages Injected with Multilingual Graph Knowledge
Contextualized embeddings based on large language models (LLMs) are available for various languages, but their coverage is often limited for lower resourced languages. Using LLMs for such languages is often difficult due…
Natural Language InferenceSentiment AnalysisTopic ClassificationWord Embeddings+1mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval
We present systematic efforts in building long-context multilingual text representation model (TRM) and reranker from scratch for text retrieval. We first introduce a text encoder (base size) enhanced with RoPE and unpad…
Contrastive LearningRerankingRetrievalText Retrieval+1The Model Arena for Cross-lingual Sentiment Analysis: A Comparative Study in the Era of Large Language Models
Sentiment analysis serves as a pivotal component in Natural Language Processing (NLP). Advancements in multilingual pre-trained models such as XLM-R and mT5 have contributed to the increasing interest in cross-lingual se…
Cross-Lingual TransferSentiment AnalysisXLM-RMedical Spoken Named Entity Recognition
Spoken Named Entity Recognition (NER) aims to extract named entities from speech and categorise them into types like person, location, organization, etc. In this work, we present VietMed-NER - the first spoken NER datase…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+1Multilingual Large Language Models and Curse of Multilinguality
Multilingual Large Language Models (LLMs) have gained large popularity among Natural Language Processing (NLP) researchers and practitioners. These models, trained on huge datasets, show proficiency across various langua…
DecoderXLM-RExploring Alignment in Shared Cross-lingual Spaces
Despite their remarkable ability to capture linguistic nuances across diverse languages, questions persist regarding the degree of alignment between languages in multilingual embeddings. Drawing inspiration from research…
Machine Translationnamed-entity-recognitionNamed Entity RecognitionSentiment Analysis+1