Positional Description for Numerical Normalization
We present a Positional Description Scheme (PDS) tailored for digit sequences, integrating placeholder value information for each digit. Given the structural limitations of subword tokenization algorithms, language models encounter critical Text Normalization (TN) challenges when handling numerical tasks. Our schema addresses this challenge through straightforward pre-processing, preserving the model architecture while significantly simplifying number normalization, rendering the problem tractable. This simplifies the task and facilitates more compact production-ready models capable of learning from smaller datasets. Furthermore, our investigations reveal that PDS enhances the arithmetic processing capabilities of language models, resulting in a relative accuracy improvement of 23% to 51% on complex arithmetic tasks. We demonstrate that PDS effectively mitigates fatal numerical normalization errors in neural models, requiring only a modest amount of training data without rule-based Finite State Transducers (FST). We demonstrate that PDS is essential for both the Text-To-Speech and Speech Recognition text processing, enabling effective TN under production constraints.
Code (0)
등록된 구현이 없습니다.
Tasks
speech-recognitionSpeech RecognitionText Normalizationtext-to-speechText to SpeechSimilar Papers 제목 키워드 기반
Group-wise normalization in differential abundance analysis of microbiome samples
A key challenge in differential abundance analysis of microbial samples is that the counts for each sample are compositional, resulting in biased comparisons of the absolute abundance across study groups. Normalization-b…
What functions can Graph Neural Networks compute on random graphs? The role of Positional Encoding
We aim to deepen the theoretical understanding of Graph Neural Networks (GNNs) on large graphs, with a focus on their expressive power. Existing analyses relate this notion to the graph isomorphism problem, which is most…
Graph ClassificationComparing the Decision-Making Mechanisms by Transformers and CNNs via Explanation Methods
In order to gain insights about the decision-making of different visual recognition backbones, we propose two methodologies, sub-explanation counting and cross-testing, that systematically applies deep explanation algori…
Decision MakingA Generalization of Transformer Networks to Graphs
We propose a generalization of transformer neural network architecture for arbitrary graphs. The original transformer was designed for Natural Language Processing (NLP), which operates on fully connected graphs represent…
Graph RegressionInductive BiasKnowledge GraphsLink Prediction+1Learning to Generate Compositional Color Descriptions
The production of color language is essential for grounded language generation. Color descriptions have many challenging properties: they can be vague, compositionally complex, and denotationally rich. We present an effe…
Language ModelingLanguage ModellingText Generation