Improving Korean NLP Tasks with Linguistically Informed Subword Tokenization and Sub-character Decomposition
We introduce a morpheme-aware subword tokenization method that utilizes sub-character decomposition to address the challenges of applying Byte Pair Encoding (BPE) to Korean, a language characterized by its rich morphology and unique writing system. Our approach balances linguistic accuracy with computational efficiency in Pre-trained Language Models (PLMs). Our evaluations show that this technique achieves good performances overall, notably improving results in the syntactic task of NIKL-CoLA. This suggests that integrating morpheme type information can enhance language models' syntactic and semantic capabilities, indicating that adopting more linguistic insights can further improve performance beyond standard morphological analysis.
Code (1)
Tasks
CoLAComputational EfficiencyMorphological AnalysisSimilar Papers 제목 키워드 기반
SCRIPT: A Subcharacter Compositional Representation Injection Module for Korean Pre-Trained Language Models
Korean is a morphologically rich language with a featural writing system in which each character is systematically composed of subcharacter units known as Jamo. These subcharacters not only determine the visual structure…
Natural Language UnderstandingShould we find another model?: Improving Neural Machine Translation Performance with ONE-Piece Tokenization Method without Model Modification
Most of the recent Natural Language Processing(NLP) studies are based on the Pretrain-Finetuning Approach (PFA), but in small and medium-sized enterprises or companies with insufficient hardware there are many limitation…
Machine TranslationmodelTranslationTokenization Disparities as Infrastructure Bias: How Subword Systems Create Inequities in LLM Access and Efficiency
Tokenization disparities pose a significant barrier to achieving equitable access to artificial intelligence across linguistically diverse populations. This study conducts a large-scale cross-linguistic evaluation of tok…
Vacillating Human Correlation of SacreBLEU in Unprotected Languages
SacreBLEU, by incorporating a text normalizing step in the pipeline, has become a rising automatic evaluation metric in recent MT studies. With agglutinative languages such as Korean, however, the lexical-level metric ca…
Linguistically Informed Tokenization Improves ASR for Underresourced Languages
Automatic speech recognition (ASR) is a crucial tool for linguists aiming to perform a variety of language documentation tasks. However, modern ASR systems use data-hungry transformer architectures, rendering them genera…
Speech Recognition