Multi-task learning for historical text normalization: Size matters
Historical text normalization suffers from small datasets that exhibit high variance, and previous work has shown that multi-task learning can be used to leverage data from related problems in order to obtain more robust models. Previous work has been limited to datasets from a specific language and a specific historical period, and it is not clear whether results generalize. It therefore remains an open problem, when historical text normalization benefits from multi-task learning. We explore the benefits of multi-task learning across 10 different datasets, representing different languages and periods. Our main finding{---}contrary to what has been observed for other NLP tasks{---}is that multi-task learning mainly works when target task data is very scarce.
Code (0)
등록된 구현이 없습니다.
Tasks
Grammatical Error CorrectionMulti-Task LearningText GenerationText NormalizationSimilar Papers 제목 키워드 기반
Few-Shot and Zero-Shot Learning for Historical Text Normalization
Historical text normalization often relies on small training datasets. Recent work has shown that multi-task learning can lead to significant improvements by exploiting synergies with related datasets, but there has been…
LemmatizationMulti-Task LearningText NormalizationZero-Shot LearningImproving historical spelling normalization with bi-directional LSTMs and multi-task learning
Natural-language processing of historical documents is complicated by the abundance of variant spellings and lack of annotated data. A common approach is to normalize the spelling of historical words to modern forms. We …
Multi-Task LearningApplying the Transformer to Character-level Transduction
The transformer has been shown to outperform recurrent neural network-based sequence-to-sequence models in various word-level NLP tasks. Yet for character-level transduction tasks, e.g. morphological inflection generatio…
Grapheme-to-Phoneme ConversionMorphological InflectionText NormalizationTransliterationSemi-supervised Contextual Historical Text Normalization
Historical text normalization, the task of mapping historical word forms to their modern counterparts, has recently attracted a lot of interest (Bollmann, 2019; Tang et al., 2018; Lusetti et al., 2018; Bollmann et al., 2…
Language ModelingLanguage ModellingText NormalizationIs text normalization relevant for classifying medieval charters?
This study examines the impact of historical text normalization on the classification of medieval charters, specifically focusing on document dating and locating. Using a data set of Middle High German charters from a di…
Document DatingText Normalization