Word segmentation granularity in Korean
This paper describes word {segmentation} granularity in Korean language processing. From a word separated by blank space, which is termed an eojeol, to a sequence of morphemes in Korean, there are multiple possible levels of word segmentation granularity in Korean. For specific language processing and corpus annotation tasks, several different granularity levels have been proposed and utilized, because the agglutinative languages including Korean language have a one-to-one mapping between functional morpheme and syntactic category. Thus, we analyze these different granularity levels, presenting the examples of Korean language processing systems for future reference. Interestingly, the granularity by separating only functional morphemes including case markers and verbal endings, and keeping other suffixes for morphological derivation results in the optimal performance for phrase structure parsing. This contradicts previous best practices for Korean language processing, which has been the de facto standard for various applications that require separating all morphemes.
Code (0)
등록된 구현이 없습니다.
Tasks
SegmentationSimilar Papers 제목 키워드 기반
Segmentation Granularity in Dependency Representations for Korean
Word Segmentation for Asian Languages: Chinese, Korean, and Japanese
We provide a detailed overview of various approaches to word segmentation of Asian Languages, specifically Chinese, Korean, and Japanese languages. For each language, approaches to deal with word segmentation differs. We…
SegmentationSCRIPT: A Subcharacter Compositional Representation Injection Module for Korean Pre-Trained Language Models
Korean is a morphologically rich language with a featural writing system in which each character is systematically composed of subcharacter units known as Jamo. These subcharacters not only determine the visual structure…
Natural Language UnderstandingStatistical word segmentation in spontaneous child-directed speech of Korean
The present study demonstrates advantages of child-directed speech (CDS) over adult-directed speech (ADS) in statistical word segmentation of spontaneous Korean. We derived phonetic input from phonemic corpus by applying…
Segmentation