The Impact of Preprocessing on Arabic-English Statistical and Neural Machine Translation
Neural networks have become the state-of-the-art approach for machine translation (MT) in many languages. While linguistically-motivated tokenization techniques were shown to have significant effects on the performance of statistical MT, it remains unclear if those techniques are well suited for neural MT. In this paper, we systematically compare neural and statistical MT models for Arabic-English translation on data preprecossed by various prominent tokenization schemes. Furthermore, we consider a range of data and vocabulary sizes and compare their effect on both approaches. Our empirical results show that the best choice of tokenization scheme is largely based on the type of model and the size of data. We also show that we can gain significant improvements using a system selection that combines the output from neural and statistical MT.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
JCT at SemEval-2022 Task 6-A: Sarcasm Detection in Tweets Written in English and Arabic using Preprocessing Methods and Word N-grams
In this paper, we describe our submissions to SemEval-2022 contest. We tackled subtask 6-A - “iSarcasmEval: Intended Sarcasm Detection In English and Arabic – Binary Classification”. We developed different models for two…
Binary ClassificationSarcasm DetectionA Recipe for Arabic-English Neural Machine Translation
In this paper, we present a recipe for building a good Arabic-English neural machine translation. We compare neural systems with traditional phrase-based systems using various parallel corpora including UN, ISI and Ummah…
Machine TranslationTranslationTw-StAR at SemEval-2018 Task 1: Preprocessing Impact on Multi-label Emotion Classification
In this paper, we describe our contribution in SemEval-2018 contest. We tackled task 1 {``}Affect in Tweets{''}, subtask E-c {``}Detecting Emotions (multi-label classification){''}. A multilabel classification system Tw-…
ClassificationEmotion ClassificationGeneral ClassificationLemmatization+4A Simple but Effective Approach to Improve Arabizi-to-English Statistical Machine Translation
A major challenge for statistical machine translation (SMT) of Arabic-to-English user-generated text is the prevalence of text written in Arabizi, or Romanized Arabic. When facing such texts, a translation system trained…
Machine TranslationTranslationTransliterationFirst Result on Arabic Neural Machine Translation
Neural machine translation has become a major alternative to widely used phrase-based statistical machine translation. We notice however that much of research on neural machine translation has focused on European languag…
Machine TranslationTranslation