paper-with-me

홈 › Papers

SiPaKosa: A Comprehensive Corpus of Canonical and Classical Buddhist Texts in Sinhala and Pali

2026-03-31 · Ranidu Gurusinghe, Nevidu Jayatilleke arxiv

SiPaKosa is a comprehensive corpus of Sinhala and Pali doctrinal texts comprising approximately 786K sentences and 9.25M words, incorporating 16 copyright-cleared historical Buddhist documents alongside the complete web-scraped Tripitaka canonical texts. The corpus was created through high-quality OCR using Google Document AI on historical manuscripts, combined with systematic web scraping of canonical repositories, followed by rigorous quality control and metadata annotation. The corpus is organised into language-specific subcorpora: Sinhala and Mixed Sinhala-Pali. We evaluate the performance of language models using ten pretrained models, with perplexity scores ranging from 1.09 to 189.67 on our corpus. This analysis shows that proprietary models significantly outperform open-source alternatives by factors of three to six times. This corpus supports the pretraining of domain-adapted language models, facilitates historical language analysis, and aids in the development of information retrieval systems for Buddhist scholarship while preserving Sinhala cultural heritage.

📄 PDF Abstract BibTeX arXiv:2603.29221

Code (0)

등록된 구현이 없습니다.

Tasks

Information RetrievalDocument AI

Similar Papers 제목 키워드 기반

MITRA: A Large-Scale Parallel Corpus and Multilingual Pretrained Language Model for Machine Translation and Semantic Retrieval for Pāli, Sanskrit, Buddhist Chinese, and Tibetan

2026-01-10 · Sebastian Nehrdich, Kurt Keutzer arxiv

Ancient Buddhist literature features frequent, yet often unannotated, textual parallels spread across diverse languages: Sanskrit, Pāli, Buddhist Chinese, Tibetan, and more. The scale of this material makes manual examin…

Machine TranslationSemantic SimilaritySemantic Retrieval

Embeddings models for Buddhist Sanskrit

2022-06-01 · LREC 2022 6 · Ligeia Lugli, Matej Martinc, Andraž Pelicon, Senja Pollak

The paper presents novel resources and experiments for Buddhist Sanskrit, broadly defined here including all the varieties of Sanskrit in which Buddhist texts have been transmitted. We release a novel corpus of Buddhist …

Semantic SimilaritySemantic Textual SimilarityTransfer LearningWord Similarity

From Vajrayana Tara to Bengali Baul: A Computational Study of Lexical Transmission Across Buddhist, Shakta, and Vaishnava Traditions in Bengal

2026-06-25 · Joy Bose arxiv

We present a computational corpus study of vocabulary relationships across eight tradition layers of Bengali and Sanskrit devotional literature spanning the 8th to 19th centuries, encompassing Buddhist Vajrayana, Shakta …

Transliteration Extraction from Classical Chinese Buddhist Literature Using Conditional Random Fields

2013-11-01 · PACLIC 2013 11 · Yu-Chun Wang, Richard Tzong-Han Tsai
Transliteration

Transliteration Extraction from Classical Chinese Buddhist Literature Using Conditional Random Fields with Language Models

2014-09-01 · ROCLINGIJCLCLP 2014 9 · Yu-Chun Wang, Karol Chia-Tien Chang, Richard Tzong-Han Tsai, Jieh Hsiang
Transliteration