paper-with-me

홈 › Papers

Domain Adaptation for Arabic Cross-Domain and Cross-Dialect Sentiment Analysis from Contextualized Word Embedding

2021-06-01 · NAACL 2021 4 · Abdellah El Mekki, Abdelkader El Mahdaouy, Ismail Berrada, Ahmed Khoumsi

Finetuning deep pre-trained language models has shown state-of-the-art performances on a wide range of Natural Language Processing (NLP) applications. Nevertheless, their generalization performance drops under domain shift. In the case of Arabic language, diglossia makes building and annotating corpora for each dialect and/or domain a more challenging task. Unsupervised Domain Adaptation tackles this issue by transferring the learned knowledge from labeled source domain data to unlabeled target domain data. In this paper, we propose a new unsupervised domain adaptation method for Arabic cross-domain and cross-dialect sentiment analysis from Contextualized Word Embedding. Several experiments are performed adopting the coarse-grained and the fine-grained taxonomies of Arabic dialects. The obtained results show that our method yields very promising results and outperforms several domain adaptation methods for most of the evaluated datasets. On average, our method increases the performance by an improvement rate of 20.8{\%} over the zero-shot transfer learning from BERT.

📄 PDF Abstract BibTeX

Code (1)

4mekki4/arabic-nlp-da 공식 구현 pytorch

Tasks

Domain AdaptationSentiment AnalysisTransfer LearningUnsupervised Domain Adaptation

Methods 이 논문이 사용한 방법론

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…
Multi-Head Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Attention 설명 없음
Adam 설명 없음
Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.
Residual Connection 설명 없음
WordPiece 설명 없음

Similar Papers 제목 키워드 기반

Figurative and Cultural Knowledge in LLMs: Investigating Cross-Domain Transfer through Fine-Tuning

2026-08-18 · Mena Attia, Mona Diab, Thamar Solorio arxiv

Figurative language is deeply culturally embedded; fluent use requires not just linguistic competence but cultural immersion. We ask whether LLMs can learn this link: does fine-tuning on cultural data improve figurative …

Leveraging Domain Adaptation and Data Augmentation to Improve Qur'anic IR in English and Arabic

2023-12-05 · Vera Pavlova

In this work, we approach the problem of Qur'anic information retrieval (IR) in Arabic and English. Using the latest state-of-the-art methods in neural IR, we research what helps to tackle this task more efficiently. Tra…

Data AugmentationDomain AdaptationInformation RetrievalLanguage Modelling+1

Resource-Aware Arabic LLM Creation: Model Adaptation, Integration, and Multi-Domain Testing

2024-12-23 · Prakash Aryan

This paper presents a novel approach to fine-tuning the Qwen2-1.5B model for Arabic language processing using Quantized Low-Rank Adaptation (QLoRA) on a system with only 4GB VRAM. We detail the process of adapting this l…

ArabicMMLUDialect IdentificationGPULanguage Modeling+5

Domain Adaptation for Arabic Machine Translation: The Case of Financial Texts

2023-09-22 · Emad A. Alghamdi, Jezia Zakraoui, Fares A. Abanmy

Neural machine translation (NMT) has shown impressive performance when trained on large-scale corpora. However, generic NMT systems have demonstrated poor performance on out-of-domain translation. To mitigate this issue,…

ArticlesBenchmarkingDomain AdaptationMachine Translation+3

MentalMARBERT: Domain-Adaptive Pre-training and Two-Stage Fine-Tuning for Arabic Mental Health Disorders Detection

2026-06-10 · Fatimah Almalki, Areej Alhothali, Lulwah Alharigy, Abdulrahman Aladeem arxiv

Detecting mental health disorders from Arabic social media text remains challenging due to dialectal variation, informal language, limited high-quality annotated resources, and severe class imbalance. While English menta…

Text Classification