Exploring Word Alignment towards an Efficient Sentence Aligner for Filipino and Cebuano Languages
Building a robust machine translation (MT) system requires a large amount of parallel corpus which is an expensive resource for low-resourced languages. The two major languages being spoken in the Philippines which are Filipino and Cebuano have an abundance in monolingual data that this study took advantage of attempting to find the best way to automatically generate parallel corpus out from monolingual corpora through the use of bitext alignment. Byte-pair encoding was applied in an attempt to optimize the alignment of the source and target texts. Results have shown that alignment was best achieved without segmenting the tokens. Itermax alignment score is best for short-length sentences and match or argmax alignment score are best for long-length sentences.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationSentenceTranslationWord AlignmentSimilar Papers 제목 키워드 기반
Third-Party Aligner for Neural Word Alignments
Word alignment is to find translationally equivalent words between source and target sentences. Previous work has demonstrated that self-training can achieve competitive word alignment results. In this paper, we propose …
Language ModelingLanguage ModellingWord AlignmentTowards Stress-Aware Sentence-Level Filipino G2P With Weakly-Supervised ByT5 Fine-Tuning
Grapheme-to-phoneme conversion (G2P) refers to the task of converting a sequence of graphemes to a corresponding sequence of phonemes. While Filipino G2P is fairly straightforward due to its shallow orthography, the incl…
Multilingual Sentence Transformer as A Multilingual Word Aligner
Multilingual pretrained language models (mPLMs) have shown their effectiveness in multilingual word alignment induction. However, these methods usually start from mBERT or XLM-R. In this paper, we investigate whether mul…
SentenceWord AlignmentXLM-RAnnotated Corpora for Word Alignment between Japanese and English and its Evaluation with MAP-based Word Aligner
This paper presents two annotated corpora for word alignment between Japanese and English. We annotated on top of the IWSLT-2006 and the NTCIR-8 corpora. The IWSLT-2006 corpus is in the domain of travel conversation whil…
Machine TranslationSentenceWord AlignmentOmniAlign: A Unified Multilingual Aligner for Word and Sentence Alignment
Cross-lingual sequence alignment is fundamental for building and exploiting parallel corpora, spanning mappings from documents and sentences down to words and subwords. Existing tools, however, typically specialize in a …
Long-Context UnderstandingSelf-Supervised LearningWord Alignment