Word Segmentation and Morphological Parsing for Sanskrit
We describe our participation in the Word Segmentation and Morphological Parsing (WSMP) for Sanskrit hackathon. We approach the word segmentation task as a sequence labelling task by predicting edit operations from which segmentations are derived. We approach the morphological analysis task by predicting morphological tags and rules that transform inflected words into their corresponding stems. Also, we propose an end-to-end trainable pipeline model for joint segmentation and morphological analysis. Our model performed best in the joint segmentation and analysis subtask (80.018 F1 score) and performed second best in the individual subtasks (segmentation: 96.189 F1 score / analysis: 69.180 F1 score). Finally, we analyse errors made by our models and suggest future work and possible improvements regarding data and evaluation.
Code (1)
Tasks
Morphological AnalysisSegmentationSimilar Papers 제목 키워드 기반
One Model is All You Need: ByT5-Sanskrit, a Unified Model for Sanskrit NLP Tasks
Morphologically rich languages are notoriously challenging to process for downstream NLP applications. This paper presents a new pretrained language model, ByT5-Sanskrit, designed for NLP applications involving the morph…
AllDependency ParsingInformation RetrievalLanguage Modelling+4Free as in Free Word Order: An Energy Based Model for Word Segmentation and Morphological Tagging in Sanskrit
The configurational information in sentences of a free word order language such as Sanskrit is of limited use. Thus, the context of the entire sentence will be desirable even for basic processing tasks such as word segme…
Morphological TaggingSegmentationSentenceStructured PredictionSHR++: An Interface for Morpho-syntactic Annotation of Sanskrit Corpora
We propose a web-based annotation framework, SHR++, for morpho-syntactic annotation of corpora in Sanskrit. SHR++ is designed to generate annotations for the word-segmentation, morphological parsing and dependency analys…
Decision MakingSegmentationvalidLinguistically-Informed Neural Architectures for Lexical, Syntactic and Semantic Tasks in Sanskrit
The primary focus of this thesis is to make Sanskrit manuscripts more accessible to the end-users through natural language technologies. The morphological richness, compounding, free word orderliness, and low-resource na…
Dependency ParsingMachine TranslationQuestion AnsweringKeep it Surprisingly Simple: A Simple First Order Graph Based Parsing Model for Joint Morphosyntactic Parsing in Sanskrit
Morphologically rich languages seem to benefit from joint processing of morphology and syntax, as compared to pipeline architectures. We propose a graph-based model for joint morphological parsing and dependency parsing …
Dependency ParsingGraph GenerationStructured Prediction