Alleviating Digitization Errors in Named Entity Recognition for Historical Documents
This paper tackles the task of named entity recognition (NER) applied to digitized historical texts obtained from processing digital images of newspapers using optical character recognition (OCR) techniques. We argue that the main challenge for this task is that the OCR process leads to misspellings and linguistic errors in the output text. Moreover, historical variations can be present in aged documents, which can impact the performance of the NER process. We conduct a comparative evaluation on two historical datasets in German and French against previous state-of-the-art models, and we propose a model based on a hierarchical stack of Transformers to approach the NER task for historical data. Our findings show that the proposed model clearly improves the results on both historical datasets, and does not degrade the results for modern datasets.
Code (1)
Tasks
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NEROptical Character RecognitionOptical Character Recognition (OCR)Similar Papers 제목 키워드 기반
Named Entity Recognition in Historical Italian: The Case of Giacomo Leopardi's Zibaldone
The increased digitization of world's textual heritage poses significant challenges for both computer science and literary studies. Overall, there is an urgent need of computational techniques able to adapt to the challe…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NERLights, Camera, Action! A Framework to Improve NLP Accuracy over OCR documents
Document digitization is essential for the digital transformation of our societies, yet a crucial step in the process, Optical Character Recognition (OCR), is still not perfect. Even commercial OCR systems can produce qu…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+2Named Entity Recognition - Is There a Glass Ceiling?
Recent developments in Named Entity Recognition (NER) have resulted in better and better models. However, is there a glass ceiling? Do we know which types of errors are still hard or even impossible to correct? In this p…
BIG-bench Machine Learningnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+1Improving Named Entity Recognition in Telephone Conversations via Effective Active Learning with Human in the Loop
Telephone transcription data can be very noisy due to speech recognition errors, disfluencies, etc. Not only that annotating such data is very challenging for the annotators, but also such data may have lots of annotatio…
Active Learningnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+2