Papers Text Normalization
“Text Normalization” 태그가 달린 논문 145편 · 필터 해제
RNN-T For Latency Controlled ASR With Improved Beam Search
Neural transducer-based systems such as RNN Transducers (RNN-T) for automatic speech recognition (ASR) blend the individual components of a traditional hybrid ASR systems (acoustic model, language model, punctuation mode…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Computational EfficiencyLanguage Modeling+4Dialect Text Normalization to Normative Standard Finnish
We compare different LSTMs and transformer models in terms of their effectiveness in normalizing dialectal Finnish into the normative standard Finnish. As dialect is the common way of communication for people online in F…
Text NormalizationBenefits of Data Augmentation for NMT-based Text Normalization of User-Generated Content
One of the most persistent characteristics of written user-generated content (UGC) is the use of non-standard words. This characteristic contributes to an increased difficulty to automatically process and analyze UGC. Te…
Data AugmentationDecoderMachine TranslationNMT+2Unsupervised Neologism Normalization Using Embedding Space Mapping
This paper presents an approach for detecting and normalizing neologisms in social media content. Neologisms refer to recent expressions that are specific to certain entities or events and are being increasingly used by …
Natural Language UnderstandingText NormalizationAmharic Text Normalization with Sequence-to-Sequence Models
All areas of language and speech technology, directly or indirectly, require handling of real text. In addition to ordinary words and names, the real text contains non-standard words (NSWs), including numbers, abbreviati…
DecoderText NormalizationComparing MT Approaches for Text Normalization
One of the main characteristics of social media data is the use of non-standard language. Since NLP tools have been trained on traditional text material their performance drops when applied to social media data. One way …
Data AugmentationLanguage ModelingLanguage ModellingMachine Translation+3Give It a Shot: Few-shot Learning to Normalize ADR Mentions in Social Media Posts
This paper describes the system that team MYTOMORROWS-TU DELFT developed for the 2019 Social Media Mining for Health Applications (SMM4H) Shared Task 3, for the end-to-end normalization of ADR tweet mentions to their cor…
Entity LinkingFew-Shot LearningText NormalizationWord EmbeddingsHistorical Text Normalization with Delayed Rewards
Training neural sequence-to-sequence models with simple token-level log-likelihood is now a standard approach to historical text normalization, albeit often outperformed by phrase-based models. Policy gradient training e…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Text NormalizationNormalizing Non-canonical Turkish Texts Using Machine Translation Approaches
With the growth of the social web, user-generated text data has reached unprecedented sizes. Non-canonical text normalization provides a way to exploit this as a practical source of training data for language processing …
Machine TranslationText NormalizationTranslationNeural Text Normalization with Subword Units
Text normalization (TN) is an important step in conversational systems. It converts written text to its spoken form to facilitate speech recognition, natural language understanding and text-to-speech synthesis. Finite st…
Machine TranslationNatural Language Understandingspeech-recognitionSpeech Recognition+6Neural Models of Text Normalization for Speech Applications
Machine learning, including neural network techniques, have been applied to virtually every domain in natural language processing. One problem that has been somewhat resistant to effective machine learning solutions is t…
BIG-bench Machine LearningSpeech SynthesisText Normalizationtext-to-speech+2Adapting Sequence to Sequence models for Text Normalization in Social Media
Social media offer an abundant source of valuable raw data, however informal writing can quickly become a bottleneck for many natural language processing (NLP) tasks. Off-the-shelf tools are usually trained on formal tex…
DecoderLexical NormalizationText NormalizationA Large-Scale Comparison of Historical Text Normalization Systems
There is no consensus on the state-of-the-art approach to historical text normalization. Many techniques have been proposed, including rule-based methods, distance metrics, character-based statistical machine translation…
DecoderMachine TranslationText NormalizationTranslationMultilevel Text Normalization with Sequence-to-Sequence Networks and Multisource Learning
We define multilevel text normalization as sequence-to-sequence processing that transforms naturally noisy text into a sequence of normalized units of meaning (morphemes) in three steps: 1) writing normalization, 2) lemm…
DecoderLanguage ModelingLanguage ModellingLemmatization+1Few-Shot and Zero-Shot Learning for Historical Text Normalization
Historical text normalization often relies on small training datasets. Recent work has shown that multi-task learning can lead to significant improvements by exploiting synergies with related datasets, but there has been…
LemmatizationMulti-Task LearningText NormalizationZero-Shot LearningA Character-Level Approach to the Text Normalization Problem Based on a New Causal Encoder
Text normalization is a ubiquitous process that appears as the first step of many Natural Language Processing problems. However, previous Deep Learning approaches have suffered from so-called silly errors, which are unde…
DecoderText NormalizationUtilizing Character and Word Embeddings for Text Normalization with Sequence-to-Sequence Models
Text normalization is an important enabling technology for several NLP tasks. Recently, neural-network-based approaches have outperformed well-established models in this task. However, in languages other than English, th…
Text NormalizationWord EmbeddingsEncoder-Decoder Methods for Text Normalization
Text normalization is the task of mapping non-canonical language, typical of speech transcription and computer-mediated communication, to a standardized writing. It is an up-stream task necessary to enable the subsequent…
DecoderMachine TranslationText NormalizationTranslationFast and Accurate Reordering with ITG Transition RNN
Attention-based sequence-to-sequence neural network models learn to jointly align and translate. The quadratic-time attention mechanism is powerful as it is capable of handling arbitrary long-distance reordering, but com…
DecoderFeature EngineeringMachine TranslationMorphological Inflection+3Multi-task learning for historical text normalization: Size matters
Historical text normalization suffers from small datasets that exhibit high variance, and previous work has shown that multi-task learning can be used to leverage data from related problems in order to obtain more robust…
Grammatical Error CorrectionMulti-Task LearningText GenerationText Normalization