Multi-Task Sequence Prediction For Tunisian Arabizi Multi-Level Annotation
In this paper we propose a multi-task sequence prediction system, based on recurrent neural networks and used to annotate on multiple levels an Arabizi Tunisian corpus. The annotation performed are text classification, tokenization, PoS tagging and encoding of Tunisian Arabizi into CODA* Arabic orthography. The system is learned to predict all the annotation levels in cascade, starting from Arabizi input. We evaluate the system on the TIGER German corpus, suitably converting data to have a multi-task problem, in order to show the effectiveness of our neural architecture. We show also how we used the system in order to annotate a Tunisian Arabizi corpus, which has been afterwards manually corrected and used to further evaluate sequence models on Tunisian data. Our system is developed for the Fairseq framework, which allows for a fast and easy use for any other sequence prediction problem.
Code (1)
Tasks
POSPOS Taggingtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
TUNIZI: a Tunisian Arabizi sentiment analysis Dataset
On social media, Arabic people tend to express themselves in their own local dialects. More particularly, Tunisians use the informal way called "Tunisian Arabizi". Analytical studies seek to explore and recognize online …
MarketingSentiment AnalysisIntroducing A large Tunisian Arabizi Dialectal Dataset for Sentiment Analysis
On various Social Media platforms, people, tend to use the informal way to communicate, or write posts and comments: their local dialects. In Africa, more than 1500 dialects and languages exist. Particularly, Tunisians t…
Language ModelingLanguage ModellingSentiment AnalysisTArC: Tunisian Arabish Corpus First complete release
In this paper we present the final result of a project on Tunisian Arabic encoded in Arabizi, the Latin-based writing system for digital conversations. The project led to the creation of two integrated and independent re…
LemmatizationPOSPOS TaggingTransliterationTArC: Tunisian Arabish Corpus, First complete release
In this paper we present the final result of a project focused on Tunisian Arabic encoded in Arabizi, the Latin-based writing system for digital conversations. The project led to the realization of two integrated and ind…
LemmatizationPOSPOS TaggingTransliterationTArC: Incrementally and Semi-Automatically Collecting a Tunisian Arabish Corpus
This article describes the constitution process of the first morpho-syntactically annotated Tunisian Arabish Corpus (TArC). Arabish, also known as Arabizi, is a spontaneous coding of Arabic dialects in Latin characters a…