Papers Text Normalization
“Text Normalization” 태그가 달린 논문 145편 · 필터 해제
Neural text normalization leveraging similarities of strings and sounds
We propose neural models that can normalize text by considering the similarities of word strings and sounds. We experimentally compared a model that considers the similarities of both word strings and sounds, a model tha…
Text NormalizationTNT: Text Normalization based Pre-training of Transformers for Content Moderation
In this work, we present a new language pre-training model TNT (Text Normalization based pre-training of Transformers) for content moderation. Inspired by the masking strategy and text normalization, TNT is developed to …
Text NormalizationService-oriented Text-to-SQL Parsing
The information retrieval from relational database requires professionals who has an understanding of structural query language such as SQL. TEXT2SQL models apply natural language inference to enable user interacting the…
Information RetrievalNatural Language InferenceRetrievalSQL Parsing+3hinglishNorm -- A Corpus of Hindi-English Code Mixed Sentences for Text Normalization
We present hinglishNorm -- a human annotated corpus of Hindi-English code-mixed sentences for text normalization task. Each sentence in the corpus is aligned to its corresponding human annotated normalized form. To the b…
SentenceText NormalizationTranslationSeq2Edits: Sequence Transduction Using Span-level Edit Operations
We propose Seq2Edits, an open-vocabulary approach to sequence editing for natural language processing (NLP) tasks with a high degree of overlap between input and output texts. In this approach, each sequence-to-sequence …
Grammatical Error CorrectionSentenceSentence FusionTAG+2Aggressive Language Detection with Joint Text Normalization via Adversarial Multi-task Learning
Aggressive language detection (ALD), detecting the abusive and offensive language in texts, is one of the crucial applications in NLP community. Most existing works treat ALD as regular classification with neural models,…
Multi-Task LearningText NormalizationInducing Language-Agnostic Multilingual Representations
Cross-lingual representations have the potential to make NLP techniques available to the vast majority of languages in the world. However, they currently require large pretraining corpora or access to typologically simil…
Cross-Lingual TransferSentenceText NormalizationXLM-R+1An improved Bayesian TRIE based model for SMS text normalization
Normalization of SMS text, commonly known as texting language, is being pursued for more than a decade. A probabilistic approach based on the Trie data structure was proposed in literature which was found to be better pe…
Text NormalizationSemi-supervised Contextual Historical Text Normalization
Historical text normalization, the task of mapping historical word forms to their modern counterparts, has recently attracted a lot of interest (Bollmann, 2019; Tang et al., 2018; Lusetti et al., 2018; Bollmann et al., 2…
Language ModelingLanguage ModellingText NormalizationNeural Multi-task Text Normalization and Sanitization with Pointer-Generator
Text normalization and sanitization are intrinsic components of Natural Language Inferences. In Information Retrieval or Dialogue Generation, normalization of user queries or utterances enhances linguistic understanding …
Dialogue GenerationInformation RetrievalRetrievalText Generation+1Normalizing Text using Language Modelling based on Phonetics and String Similarity
Social media networks and chatting platforms often use an informal version of natural text. Adversarial spelling attacks also tend to alter the input text by modifying the characters in the text. Normalizing these texts …
Language ModelingLanguage ModellingSpeech SynthesisText Normalization+4Analyzing the Quality and Stability of a Streaming End-to-End On-Device Speech Recognizer
The demand for fast and accurate incremental speech recognition increases as the applications of automatic speech recognition (ASR) proliferate. Incremental speech recognizers output chunks of partially recognized words …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech Recognition+1Applying the Transformer to Character-level Transduction
The transformer has been shown to outperform recurrent neural network-based sequence-to-sequence models in various word-level NLP tasks. Yet for character-level transduction tasks, e.g. morphological inflection generatio…
Grapheme-to-Phoneme ConversionMorphological InflectionText NormalizationTransliterationData-Driven Parametric Text Normalization: Rapidly Scaling Finite-State Transduction Verbalizers to New Languages
This paper presents a methodology for rapidly generating FST-based verbalizers for ASR and TTS systems by efficiently sourcing language-specific data. We describe a questionnaire which collects the necessary data to boot…
Text NormalizationA Spelling Correction Corpus for Multiple Arabic Dialects
Arabic dialects are the non-standard varieties of Arabic commonly spoken {--} and increasingly written on social media {--} across the Arab world. Arabic dialects do not have standard orthographies, a challenge for natur…
Spelling CorrectionText NormalizationDeveloping Resources for Automated Speech Processing of Quebec French
The analysis of the structure of speech nearly always rests on the alignment of the speech recording with a phonetic transcription. Nowadays several tools can perform this speech segmentation automatically. However, none…
SegmentationText NormalizationBurmese Speech Corpus, Finite-State Text Normalization and Pronunciation Grammars with an Application to Text-to-Speech
This paper introduces an open-source crowd-sourced multi-speaker speech corpus along with the comprehensive set of finite-state transducer (FST) grammars for performing text normalization for the Burmese (Myanmar) langua…
Text Normalizationtext-to-speechText to SpeechScalable Multilingual Frontend for TTS
This paper describes progress towards making a Neural Text-to-Speech (TTS) Frontend that works for many languages and can be easily extended to new languages. We take a Machine Translation (MT) inspired approach to const…
ChunkingMachine TranslationSentenceText Normalization+3How to Evaluate Word Representations of Informal Domain?
Diverse word representations have surged in most state-of-the-art natural language processing (NLP) applications. Nevertheless, how to efficiently evaluate such word embeddings in the informal domain such as Twitter or f…
Text NormalizationWord EmbeddingsA hybrid text normalization system using multi-head self-attention for mandarin
In this paper, we propose a hybrid text normalization system using multi-head self-attention. The system combines the advantages of a rule-based model and a neural model for text preprocessing tasks. Previous studies in …
SentenceText Normalization