Learning from Relatives: Unified Dialectal Arabic Segmentation
Arabic dialects do not just share a common koin{\'e}, but there are shared pan-dialectal linguistic phenomena that allow computational models for dialects to learn from each other. In this paper we build a unified segmentation model where the training data for different dialects are combined and a single model is trained. The model yields higher accuracies than dialect-specific models, eliminating the need for dialect identification before segmentation. We also measure the degree of relatedness between four major Arabic dialects by testing how a segmentation model trained on one dialect performs on the other dialects. We found that linguistic relatedness is contingent with geographical proximity. In our experiments we use SVM-based ranking and bi-LSTM-CRF sequence labeling.
Code (0)
등록된 구현이 없습니다.
Tasks
Dialect IdentificationInformation RetrievalMachine TranslationSegmentationSimilar Papers 제목 키워드 기반
Morphology-aware Word-Segmentation in Dialectal Arabic Adaptation of Neural Machine Translation
Parallel corpora available for building machine translation (MT) models for dialectal Arabic (DA) are rather limited. The scarcity of resources has prompted the use of Modern Standard Arabic (MSA) abundant resources to c…
Machine TranslationSegmentationTranslationDialectalArabicMMLU: Benchmarking Dialectal Capabilities in Arabic and Multilingual Language Models
We present DialectalArabicMMLU, a new benchmark for evaluating the performance of large language models (LLMs) across Arabic dialects. While recently developed Arabic and multilingual benchmarks have advanced LLM evaluat…
Hierarchical Aggregation of Dialectal Data for Arabic Dialect Identification
Arabic is a collection of dialectal variants that are historically related but significantly different. These differences can be seen across regions, countries, and even cities in the same countries. Previous work on Ara…
Dialect IdentificationLinear Semantic Segmentation for Low-Resource Spoken Dialects
Semantic segmentation is a core component of discourse analysis, yet existing models are primarily developed and evaluated on high-resource written text, limiting their effectiveness on low-resource spoken varieties. In …
Semantic SegmentationHabibi: Laying the Open-Source Foundation of Unified-Dialectal Arabic Speech Synthesis
Arabic spans over 30 spoken varieties, yet no open-source text-to-speech system unifies them. Key barriers include substantial cross-dialect lexical and phonological divergence, scarce synthesis-grade data, and the absen…
Speech Synthesis