Constructing a Bilingual Hadith Corpus Using a Segmentation Tool
This article describes the process of gathering and constructing a bilingual parallel corpus of Islamic Hadith, which is the set of narratives reporting different aspects of the prophet Muhammad{'}s life. The corpus data is gathered from the six canonical Hadith collections using a custom segmentation tool that automatically segments and annotates the two Hadith components with 92{\%} accuracy. This Hadith segmenter minimises the costs of language resource creation and produces consistent results independently from previous knowledge and experiences that usually influence human annotators. The corpus includes more than 10M tokens and will be freely available via the LREC repository.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Automatic Hadith Segmentation using PPM Compression
In this paper we explore the use of Prediction by partial matching (PPM) compression based to segment Hadith into its two main components (Isnad and Matan). The experiments utilized the PPMD variant of the PPM, showing t…
SegmentationText Segmentation Using N-grams to Annotate Hadith Corpus
Challenging the Transformer-based models with a Classical Arabic dataset: Quran and Hadith
Transformer-based models showed near-perfect results on several downstream tasks. However, their performance on classical Arabic texts is largely unexplored. To fill this gap, we evaluate monolingual, bilingual, and mult…
Speech Segmentation Optimization using Segmented Bilingual Speech Corpus for End-to-end Speech Translation
Speech segmentation, which splits long speech into short segments, is essential for speech translation (ST). Popular VAD tools like WebRTC VAD have generally relied on pause-based segmentation. Unfortunately, pauses in s…
Binary ClassificationSegmentationSentenceTranslationImproving Patent Translation using Bilingual Term Extraction and Re-tokenization for Chinese--Japanese
Unlike European languages, many Asian languages like Chinese and Japanese do not have typographic boundaries in written system. Word segmentation (tokenization) that break sentences down into individual words (tokens) is…
Chinese Word SegmentationMachine TranslationSegmentationTerm Extraction+1