Papers XLM-R
“XLM-R” 태그가 달린 논문 221편 · 필터 해제
Evaluating the Effectiveness of Pre-trained Language Models in Predicting the Helpfulness of Online Product Reviews
Businesses and customers can gain valuable information from product reviews. The sheer number of reviews often necessitates ranking them based on their potential helpfulness. However, only a few reviews ever receive any …
Feature EngineeringXLM-RModeling Sequential Sentence Relation to Improve Cross-lingual Dense Retrieval
Recently multi-lingual pre-trained language models (PLM) such as mBERT and XLM-R have achieved impressive strides in cross-lingual dense retrieval. Despite its successes, they are general-purpose PLM while the multilingu…
RelationRepresentation LearningRetrievalSentence+1LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domain
Lately, propelled by the phenomenal advances around the transformer architecture, the legal NLP field has enjoyed spectacular growth. To measure progress, well curated and challenging benchmarks are crucial. However, mos…
XLM-RMultilingual Sentence Transformer as A Multilingual Word Aligner
Multilingual pretrained language models (mPLMs) have shown their effectiveness in multilingual word alignment induction. However, these methods usually start from mBERT or XLM-R. In this paper, we investigate whether mul…
SentenceWord AlignmentXLM-RXLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models
Large multilingual language models typically rely on a single vocabulary shared across 100+ languages. As these models have increased in parameter count and depth, vocabulary size has remained largely unchanged. This \te…
Language ModelingLanguage Modellingnamed-entity-recognitionNamed Entity Recognition+4ViHOS: Hate Speech Spans Detection for Vietnamese
The rise in hateful and offensive language directed at other users is one of the adverse side effects of the increased use of social networking platforms. This could make it difficult for human moderators to review tagge…
Hate Span IdentificationSequence-to-sequence Language ModelingToxic Spans DetectionVietnamese Hate Speech Detection+2Integrating Semantic Information into Sketchy Reading Module of Retro-Reader for Vietnamese Machine Reading Comprehension
Machine Reading Comprehension has become one of the most advanced and popular research topics in the fields of Natural Language Processing in recent years. The classification of answerability questions is a relatively si…
Machine Reading ComprehensionReading ComprehensionVietnamese Machine Reading ComprehensionXLM-RDAMP: Doubly Aligned Multilingual Parser for Task-Oriented Dialogue
Modern virtual assistants use internal semantic parsing engines to convert user utterances to actionable commands. However, prior work has demonstrated that semantic parsing is a difficult multilingual transfer task with…
Semantic ParsingXLM-RVTCC-NLP at NL4Opt competition subtask 1: An Ensemble Pre-trained language models for Named Entity Recognition
We propose a combined three pre-trained language models (XLM-R, BART, and DeBERTa-V3) as an empower of contextualized embedding for named entity recognition. Our model achieves a 92.9% F1 score on the test set and ranks …
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)XLM-RTowards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languages
Building Natural Language Understanding (NLU) capabilities for Indic languages, which have a collective speaker base of more than one billion speakers is absolutely crucial. In this work, we aim to improve the NLU capabi…
Natural Language UnderstandingXLM-RPunctuation Restoration for Singaporean Spoken Languages: English, Malay, and Mandarin
This paper presents the work of restoring punctuation for ASR transcripts generated by multilingual ASR systems. The focus languages are English, Mandarin, and Malay which are three of the most popular languages in Singa…
Language ModellingPunctuation Restorationslot-fillingSlot Filling+1Languages You Know Influence Those You Learn: Impact of Language Characteristics on Multi-Lingual Text-to-Text Transfer
Multi-lingual language models (LM), such as mBERT, XLM-R, mT5, mBART, have been remarkably successful in enabling natural language tasks in low-resource languages through cross-lingual transfer from high-resource ones. I…
Cross-Lingual TransferXLM-RCompressing Cross-Lingual Multi-Task Models at Qualtrics
Experience management is an emerging business area where organizations focus on understanding the feedback of customers and employees in order to improve their end-to-end experiences. This results in a unique set of mach…
ManagementModel Compressiontext-classificationText Classification+1X$^2$-VLM: All-In-One Pre-trained Model For Vision-Language Tasks
Vision language pre-training aims to learn alignments between vision and language from a large amount of data. Most existing methods only learn image-text alignments. Some others utilize pre-trained object detectors to l…
AllCross-Modal RetrievalImage CaptioningText to Video Retrieval+6L3Cube-HindBERT and DevBERT: Pre-Trained BERT Transformer models for Devanagari based Hindi and Marathi Languages
The monolingual Hindi BERT models currently available on the model hub do not perform better than the multi-lingual models on downstream tasks. We present L3Cube-HindBERT, a Hindi BERT model pre-trained on Hindi monoling…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)text-classification+2AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities
In this work, we present a conceptually simple and effective method to train a strong bilingual/multilingual multimodal representation model. Starting from the pre-trained multimodal representation model CLIP released by…
Contrastive LearningCross-Modal RetrievalImage ClassificationImage Retrieval+9Collateral facilitation in humans and language models
Are the predictions of humans and language models affected by similar things? Research suggests that while comprehending language, humans make predictions about upcoming words, with more predictable words being processed…
XLM-RIITD at the WANLP 2022 Shared Task: Multilingual Multi-Granularity Network for Propaganda Detection
We present our system for the two subtasks of the shared task on propaganda detection in Arabic, part of WANLP'2022. Subtask 1 is a multi-label classification problem to find the propaganda techniques used in a given twe…
Multi-Label ClassificationMUlTI-LABEL-ClASSIFICATIONPropaganda detectionXLM-RImproving Bilingual Lexicon Induction with Cross-Encoder Reranking
Bilingual lexicon induction (BLI) with limited bilingual supervision is a crucial yet challenging task in multilingual NLP. Current state-of-the-art BLI methods rely on the induction of cross-lingual word embeddings (CLW…
Bilingual Lexicon InductionCross Encoder RerankingCross-Lingual Word EmbeddingsMachine Translation+9Legal-Tech Open Diaries: Lesson learned on how to develop and deploy light-weight models in the era of humongous Language Models
In the era of billion-parameter-sized Language Models (LMs), start-ups have to follow trends and adapt their technology accordingly. Nonetheless, there are open challenges since the development and deployment of large mo…
Knowledge DistillationModel CompressionQuantizationXLM-R