Proper Name Diacritization for Arabic Wikipedia: A Benchmark Dataset
Proper names in Arabic Wikipedia are frequently undiacritized, creating ambiguity in pronunciation and interpretation, especially for transliterated named entities of foreign origin. While transliteration and diacritization have been well-studied separately in Arabic NLP,their intersection remains underexplored. In this paper, we introduce a new manually diacritized dataset of Arabic proper names of various origins with their English Wikipedia equivalent glosses, and present the challenges and guidelines we followed to create it. We benchmark GPT-4o on the task of recovering full diacritization given the undiacritized Arabic and English forms, and analyze its performance. Achieving 73% accuracy, our results underscore both the difficulty of the task and the need for improved models and resources. We release our dataset to facilitate further research on Arabic Wikipedia proper name diacritization.
Code (0)
등록된 구현이 없습니다.
Tasks
TransliterationSimilar Papers 제목 키워드 기반
Diacritization of Maghrebi Arabic Sub-Dialects
Diacritization process attempt to restore the short vowels in Arabic written text; which typically are omitted. This process is essential for applications such as Text-to-Speech (TTS). While diacritization of Modern Stan…
text-to-speechText to SpeechMore Data, Fewer Diacritics: Scaling Arabic TTS
Arabic Text-to-Speech (TTS) research has been hindered by the availability of both publicly available training data and accurate Arabic diacritization models. In this paper, we address the limitation by exploring Arabic …
Speech RecognitionActivity DetectionArabic Diacritization: Stats, Rules, and Hacks
In this paper, we present a new and fast state-of-the-art Arabic diacritizer that guesses the diacritics of words and then their case endings. We employ a Viterbi decoder at word-level with back-off to stem, morphologica…
DecoderPart-Of-Speech TaggingTransliterationWord Sense DisambiguationArabic Text Diacritization In The Age Of Transfer Learning: Token Classification Is All You Need
Automatic diacritization of Arabic text involves adding diacritical marks (diacritics) to the text. This task poses a significant challenge with noteworthy implications for computational processing and comprehension. In …
AllArabic Text Diacritizationtoken-classificationToken Classification+1A System for Diacritizing Four Varieties of Arabic
Short vowels, aka diacritics, are more often omitted when writing different varieties of Arabic including Modern Standard Arabic (MSA), Classical Arabic (CA), and Dialectal Arabic (DA). However, diacritics are required t…
Feature Engineeringtext-to-speechText to Speech