An Evaluation of Neural Machine Translation Models on Historical Spelling Normalization

2018-06-13 · COLING 2018 8 · Gongbo Tang, Fabienne Cap, Eva Pettersson, Joakim Nivre

In this paper, we apply different NMT models to the problem of historical spelling normalization for five languages: English, German, Hungarian, Icelandic, and Swedish. The NMT models are at different levels, have different attention mechanisms, and different neural network architectures. Our results show that NMT models are much better than SMT models in terms of character error rate. The vanilla RNNs are competitive to GRUs/LSTMs in historical spelling normalization. Transformer models perform better only when provided with more training data. We also find that subword-level models with a small subword vocabulary are better than character-level models for low-resource languages. In addition, we propose a hybrid method which further improves the performance of historical spelling normalization.

📄 PDF Abstract BibTeX arXiv:1806.05210

Code (1)

tanggongbo/normalization-NMT 공식 구현

Tasks

Machine TranslationNMTTranslation

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

Residual Connection 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

ReLU How Do I Communicate to Expedia? How Do I Communicate to Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Live Support & Special Travel…

Normalizing Early English Letters to Present-day English Spelling

2018-08-01 · COLING 2018 8 · Mika H{\"a}m{\"a}l{\"a}inen, Tanja S{\"a}ily, Jack Rueter, J{\"o}rg Tiedemann 외

This paper presents multiple methods for normalizing the most deviant and infrequent historical spellings in a corpus consisting of personal correspondence from the 15th to the 19th century. The methods include machine t…

Machine Translation Translation

Historical German Text Normalization Using Type- and Token-Based Language Modeling

2024-09-04 · Anton Ehrmanntraut

Historic variations of spelling poses a challenge for full-text search or natural language processing on historical digitized texts. To minimize the gap between the historic orthography and contemporary spelling, usually…

Decoder Language Modeling Language Modelling Large Language Model+2

Improving historical spelling normalization with bi-directional LSTMs and multi-task learning

2016-10-25 · COLING 2016 12 · Marcel Bollmann, Anders Søgaard

Natural-language processing of historical documents is complicated by the abundance of variant spellings and lack of annotated data. A common approach is to normalize the spelling of historical words to modern forms. We …

Multi-Task Learning

Detecting spelling variants in non-standard texts

2017-04-01 · EACL 2017 4 · Fabian Barteld

Spelling variation in non-standard language, e.g. computer-mediated communication and historical texts, is usually treated as a deviation from a standard spelling, e.g. 2mr as an non-standard spelling for tomorrow. Conse…

Evaluating Inter-Annotator Agreement on Historical Spelling Normalization

2016-08-01 · WS 2016 8 · Marcel Bollmann, Stefanie Dipper, Florian Petran

Optical Character Recognition (OCR)Part-Of-Speech Tagging Semantic Role Labeling