An Unsupervised method for OCR Post-Correction and Spelling Normalisation for Finnish

2020-11-06 · NoDaLiDa 2021 5 · Quan Duong, Mika Hämäläinen, Simon Hengchen

Historical corpora are known to contain errors introduced by OCR (optical character recognition) methods used in the digitization process, often said to be degrading the performance of NLP systems. Correcting these errors manually is a time-consuming process and a great part of the automatic approaches have been relying on rules or supervised machine learning. We build on previous work on fully automatic unsupervised extraction of parallel data to train a character-based sequence-to-sequence NMT (neural machine translation) model to conduct OCR error correction designed for English, and adapt it to Finnish by proposing solutions that take the rich morphology of the language into account. Our new method shows increased performance while remaining fully unsupervised, with the added benefit of spelling normalisation. The source code and models are available on GitHub and Zenodo.

📄 PDF Abstract BibTeX arXiv:2011.03502

Code (1)

ruathudo/post-ocr-correction 공식 구현 pytorch

Tasks

Machine TranslationNMTOptical Character RecognitionOptical Character Recognition (OCR)Translation

Normalising Non-standardised Orthography in Algerian Code-switched User-generated Data

2019-11-01 · WS 2019 11 · Wafia Adouane, Jean-Philippe Bernardy, Simon Dobnik

We work with Algerian, an under-resourced non-standardised Arabic variety, for which we compile a new parallel corpus consisting of user-generated textual data matched with normalised and corrected human annotations foll…

Decoder Semantic Textual Similarity Spelling Correction

Unsupervised Context-Sensitive Spelling Correction of Clinical Free-Text with Word and Character N-Gram Embeddings

2017-08-01 · WS 2017 8 · Pieter Fivez, Simon {\v{S}}uster, Walter Daelemans

We present an unsupervised context-sensitive spelling correction method for clinical free-text that uses word and character n-gram embeddings. Our method generates misspelling replacement candidates and ranks them accord…

Spelling Correction

Automatic Normalisation of Early Modern French

2022-06-01 · LREC 2022 6 · Rachel Bawden, Jonathan Poinhos, Eleni Kogkitsidou, Philippe Gambette 외

Spelling normalisation is a useful step in the study and analysis of historical language texts, whether it is manual analysis by experts or automatic analysis using downstream natural language processing (NLP) tools. Not…

OCR and post-correction of historical Finnish texts

2017-05-01 · WS 2017 5 · Senka Drobac, Pekka Kauppinen, Krister Lind{\'e}n

Optical Character Recognition (OCR)Spelling Correction

Unsupervised Context-Sensitive Spelling Correction of English and Dutch Clinical Free-Text with Word and Character N-Gram Embeddings

2017-10-19 · Pieter Fivez, Simon Šuster, Walter Daelemans