Dialect Text Normalization to Normative Standard Finnish
We compare different LSTMs and transformer models in terms of their effectiveness in normalizing dialectal Finnish into the normative standard Finnish. As dialect is the common way of communication for people online in Finnish, such a normalization is a necessary step to improve the accuracy of the existing Finnish NLP tools that are tailored for normative Finnish text. We work on a corpus consisting of dialectal data of 23 distinct Finnish dialects. The best functioning BRNN approach lowers the initial word error rate of the corpus from 52.89 to 5.73.
Code (1)
Tasks
Text NormalizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Help from the Neighbors: Estonian Dialect Normalization Using a Finnish Dialect Generator
t
Automatic Dialect Adaptation in Finnish and its Effect on Perceived Creativity
We present a novel approach for adapting text written in standard Finnish to different dialects. We experiment with character level NMT models both by using a multi-dialectal and transfer learning approaches. The models …
NMTTransfer LearningNormalization of Different Swedish Dialects Spoken in Finland
Our study presents a dialect normalization method for different Finland Swedish dialects covering six regions. We tested 5 different models, and the best model improved the word error rate from 76.45 to 28.58. Contrary t…
Finnish Dialect Identification: The Effect of Audio and Text
Finnish is a language with multiple dialects that not only differ from each other in terms of accent (pronunciation) but also in terms of morphological forms and lexical choice. We present the first approach to automatic…
Dialect IdentificationMANorm: A Normalization Dictionary for Moroccan Arabic Dialect Written in Latin Script
Social media user-generated text is actually the main resource for many NLP tasks. This text however, does not follow the standard rules of writing. Moreover, the use of dialect such as Moroccan Arabic in written communi…