Papers XLM-R
“XLM-R” 태그가 달린 논문 221편 · 필터 해제
TEET! Tunisian Dataset for Toxic Speech Detection
The complete freedom of expression in social media has its costs especially in spreading harmful and abusive content that may induce people to act accordingly. Therefore, the need of detecting automatically such a conten…
Feature EngineeringXLM-RBoosting Transformers for Job Expression Extraction and Classification in a Low-Resource Setting
In this paper, we explore possible improvements of transformer models in a low-resource setting. In particular, we present our approaches to tackle the first two of three subtasks of the MEDDOPROF competition, i.e., the …
Transfer LearningXLM-ROn the Universality of Deep Contextual Language Models
Deep Contextual Language Models (LMs) like ELMO, BERT, and their successors dominate the landscape of Natural Language Processing due to their ability to scale across multiple tasks rapidly by pre-training a single model…
Cross-Lingual TransferXLM-RZero-Shot Cross-Lingual TransferFBERT: A Neural Transformer for Identifying Offensive Content
Transformer-based models such as BERT, XLNET, and XLM-R have achieved state-of-the-art performance across various NLP tasks including the identification of offensive language and hate speech, an important problem in soci…
Language IdentificationXLM-RNearest Neighbour Few-Shot Learning for Cross-lingual Classification
Even though large pre-trained multilingual models (e.g. mBERT, XLM-R) have led to significant performance gains on a wide range of cross-lingual NLP tasks, success on many downstream tasks still relies on the availabilit…
ClassificationFew-Shot LearningXLM-RClassification of Code-Mixed Text Using Capsule Networks
A major challenge in analysing social me-dia data belonging to languages that use non-English script is its code-mixed nature. Recentresearch has presented state-of-the-art contex-tual embedding models (both monolingual …
ClassificationXLM-RSiamese Networks for Inference in Malayalam Language Texts
Natural language inference is a method of finding inferences in language texts. Understanding the meaning of a sentence and its inference is essential in many language processing applications. In this context, we conside…
Binary ClassificationClassificationNatural Language InferenceSentence+2Cross-Lingual Text Classification of Transliterated Hindi and Malayalam
Transliteration is very common on social media, but transliterated text is not adequately handled by modern neural models for various NLP tasks. In this work, we combine data augmentation approaches with a Teacher-Studen…
BenchmarkingClassificationCross-Lingual TransferData Augmentation+6Contributions of Transformer Attention Heads in Multi- and Cross-lingual Tasks
This paper studies the relative importance of attention heads in Transformer-based models to aid their interpretability in cross-lingual and multi-lingual tasks. Prior research has found that only a few attention heads a…
XLM-RSkoltechNLP at SemEval-2021 Task 2: Generating Cross-Lingual Training Data for the Word-in-Context Task
In this paper, we present a system for the solution of the cross-lingual and multilingual word-in-context disambiguation task. Task organizers provided monolingual data in several languages, but no cross-lingual training…
Language ModelingLanguage ModellingMachine TranslationTask 2+2GlossReader at SemEval-2021 Task 2: Reading Definitions Improves Contextualized Word Embeddings
Consulting a dictionary or a glossary is a familiar way for many humans to figure out what does a word in a particular context mean. We hypothesize that a system that can select a proper definition for a particular word …
Cross-Lingual TransferLanguage ModellingTask 2Word Embeddings+3LIORI at SemEval-2021 Task 2: Span Prediction and Binary Classification approaches to Word-in-Context Disambiguation
This paper presents our approaches to SemEval-2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation task. The first approach attempted to reformulate the task as a question answering problem, while t…
Binary ClassificationData AugmentationQuestion AnsweringTask 2+1COSY: COunterfactual SYntax for Cross-Lingual Understanding
Pre-trained multilingual language models, e.g., multilingual-BERT, are widely used in cross-lingual tasks, yielding the state-of-the-art performance. However, such models suffer from a large performance gap between sourc…
counterfactualNatural Language InferencePOSQuestion Answering+1ARBERT \& MARBERT: Deep Bidirectional Transformers for Arabic
Pre-trained language models (LMs) are currently integral to many natural language processing systems. Although multilingual LMs were also introduced to serve many languages, these have limitations such as being costly at…
DiversityXLM-RApplying Occam’s Razor to Transformer-Based Dependency Parsing: What Works, What Doesn’t, and What is Really Necessary
The introduction of pre-trained transformer-based contextualized word embeddings has led to considerable improvements in the accuracy of graph-based parsers for frameworks such as Universal Dependencies (UD). However, pr…
Dependency ParsingWord EmbeddingsXLM-RRobertNLP at the IWPT 2021 Shared Task: Simple Enhanced UD Parsing for 17 Languages
This paper presents our multilingual dependency parsing system as used in the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies. Our system consists of an unfactorized biaffine classifier that operate…
Dependency ParsingXLM-R利用语义关联增强的跨语言预训练模型的译文质量评估(A Cross-language Pre-trained Model with Enhanced Semantic Connection for MT Quality Estimation)
“机器翻译质量评估(QE)虽然不需要参考译文就能进行自动评估,但它需要人工标注的评估数据进行训练。基于神经网络框架的QE为了克服人工评估数据的稀缺问题,通常包括两个阶段,首先借助大规模的平行语料学习双语对齐,然后在小规模评估数据集上进行评估建模。跨语言预训练模型可以用来代替该任务第一阶段的学习过程,因此本文首先建议一个基于XLM-R的为源/目标语言统一编码的QE模型。其次,由于大多数预训练模型是在多语言的单语数据集上构建的,因此两两语言…
XLM-RIBM MNLP IE at CASE 2021 Task 1: Multigranular and Multilingual Event Detection on Protest News
In this paper, we present the event detection models and systems we have developed for Multilingual Protest News Detection - Shared Task 1 at CASE 2021. The shared task has 4 subtasks which cover event detection at diffe…
Event DetectionLanguage ModelingLanguage ModellingXLM-RTeam “DaDeFrNi” at CASE 2021 Task 1: Document and Sentence Classification for Protest Event Detection
This paper accompanies our top-performing submission to the CASE 2021 shared task, which is hosted at the workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text. Subtasks 1 an…
ArticlesBinary ClassificationEvent DetectionSentence+2Emotion Stimulus Detection in German News Headlines
Emotion stimulus extraction is a fine-grained subtask of emotion analysis that focuses on identifying the description of the cause behind an emotion expression from a text passage (e.g., in the sentence "I am happy that …
Emotion RecognitionSentenceXLM-R