AraMUS: Pushing the Limits of Data and Model Scale for Arabic Natural Language Processing
Developing monolingual large Pre-trained Language Models (PLMs) is shown to be very successful in handling different tasks in Natural Language Processing (NLP). In this work, we present AraMUS, the largest Arabic PLM with 11B parameters trained on 529GB of high-quality Arabic textual data. AraMUS achieves state-of-the-art performances on a diverse set of Arabic classification and generative tasks. Moreover, AraMUS shows impressive few-shot learning abilities compared with the best existing Arabic PLMs.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot LearningSimilar Papers 제목 키워드 기반
How Well Do LLMs Understand Tunisian Arabic?
Large Language Models (LLMs) are the engines driving today's AI agents. The better these models understand human languages, the more natural and user-friendly the interaction with AI becomes, from everyday devices like c…
Sentiment AnalysisMunsit at NADI 2025 Shared Task 2: Pushing the Boundaries of Multidialectal Arabic ASR with Weakly Supervised Pretraining and Continual Supervised Fine-tuning
Automatic speech recognition (ASR) plays a vital role in enabling natural human-machine interaction across applications such as virtual assistants, industrial automation, customer support, and real-time transcription. Ho…
Speech RecognitionConceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts
The availability of large-scale image captioning and visual question answering datasets has contributed significantly to recent successes in vision-and-language pre-training. However, these datasets are often collected w…
Caption GenerationDiversityImage CaptioningQuestion Answering+2Lexical Induction of Morphological and Orthographic Forms for Low-Resourced Languages
In this work we address the issue of high-degree lexical sparsity for non-standard languages under severe circumstance of small resources that are considered insufficient to train recent powerful language models. We prop…
Word EmbeddingsA review of sentiment analysis research in Arabic language
Sentiment analysis is a task of natural language processing which has recently attracted increasing attention. However, sentiment analysis research has mainly been carried out for the English language. Although Arabic is…
Arabic Sentiment AnalysisMachine TranslationSentiment AnalysisTransfer Learning+1