A Simple but Effective Approach to Improve Arabizi-to-English Statistical Machine Translation
A major challenge for statistical machine translation (SMT) of Arabic-to-English user-generated text is the prevalence of text written in Arabizi, or Romanized Arabic. When facing such texts, a translation system trained on conventional Arabic-English data will suffer from extremely low model coverage. In addition, Arabizi is not regulated by any official standardization and therefore highly ambiguous, which prevents rule-based approaches from achieving good translation results. In this paper, we improve Arabizi-to-English machine translation by presenting a simple but effective Arabizi-to-Arabic transliteration pipeline that does not require knowledge by experts or native Arabic speakers. We incorporate this pipeline into a phrase-based SMT system, and show that translation quality after automatically transliterating Arabizi to Arabic yields results that are comparable to those achieved after human transliteration.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationTranslationTransliterationSimilar Papers 제목 키워드 기반
Arabizi vs LLMs: Can the Genie Understand the Language of Aladdin?
In this era of rapid technological advancements, communication continues to evolve as new linguistic phenomena emerge. Among these is Arabizi, a hybrid form of Arabic that incorporates Latin characters and numbers to rep…
Machine TranslationArabizi Detection and Conversion to Arabic
Arabizi is Arabic text that is written using Latin characters. Arabizi is used to present both Modern Standard Arabic (MSA) or Arabic dialects. It is commonly used in informal settings such as social networking sites and…
Language ModelingLanguage ModellingTransliterationA Unified Model for Arabizi Detection and Transliteration using Sequence-to-Sequence Models
While online Arabic is primarily written using the Arabic script, a Roman-script variety called Arabizi is often seen on social media. Although this representation captures the phonology of the language, it is not a one-…
TransliterationHybrid approach for transliteration of Algerian arabizi: a primary study
A hybrid approach for the transliteration of Algerian Arabizi: A primary study In this paper, we present a hybrid approach for the transliteration of the Algerian Arabizi. We define a set of rules enable us the passage f…
TransliterationLILI: A Simple Language Independent Approach for Language Identification
We introduce a generic Language Independent Framework for Linguistic Code Switch Point Detection. The system uses characters level 5-grams and word level unigram language models to train a conditional random fields (CRF)…
Language Identification