Enhance to Read Better: A Multi-Task Adversarial Network for Handwritten Document Image Enhancement
Handwritten document images can be highly affected by degradation for different reasons: Paper ageing, daily-life scenarios (wrinkles, dust, etc.), bad scanning process and so on. These artifacts raise many readability issues for current Handwritten Text Recognition (HTR) algorithms and severely devalue their efficiency. In this paper, we propose an end to end architecture based on Generative Adversarial Networks (GANs) to recover the degraded documents into a clean and readable form. Unlike the most well-known document binarization methods, which try to improve the visual quality of the degraded document, the proposed architecture integrates a handwritten text recognizer that promotes the generated document image to be more readable. To the best of our knowledge, this is the first work to use the text information while binarizing handwritten documents. Extensive experiments conducted on degraded Arabic and Latin handwritten documents demonstrate the usefulness of integrating the recognizer within the GAN architecture, which improves both the visual quality and the readability of the degraded document images. Moreover, we outperform the state of the art in H-DIBCO challenges, after fine tuning our pre-trained model with synthetically degraded Latin handwritten images, on this task.
Code (1)
Tasks
BinarizationHandwritten Text RecognitionHTRImage EnhancementSimilar Papers 제목 키워드 기반
SRQA: Synthetic Reader for Factoid Question Answering
The question answering system can answer questions from various fields and forms with deep neural networks, but it still lacks effective ways when facing multiple evidences. We introduce a new model called SRQA, which me…
Question AnsweringAdversarial multi-task learning to solve holistic semantic polishing problem of Chinese text
For the core obstacles faced by Chinese proofreading, we think that the effective holistic semantic feature representation of a sentence and its characteristics is a key. Here, we propose a novel adversarial multi-task l…
Multi-Task LearningSentenceImproving Machine Reading Comprehension via Adversarial Training
Adversarial training (AT) as a regularization method has proved its effectiveness in various tasks, such as image classification and text classification. Though there are successful applications of AT in many tasks of na…
General Classificationimage-classificationImage ClassificationMachine Reading Comprehension+3To Reverse the Gradient or Not: An Empirical Comparison of Adversarial and Multi-task Learning in Speech Recognition
Transcribed datasets typically contain speaker identity for each instance in the data. We investigate two ways to incorporate this information during training: Multi-Task Learning and Adversarial Learning. In multi-task …
Multi-Task LearningSpeaker Recognitionspeech-recognitionSpeech RecognitionMANGO: Enhancing the Robustness of VQA Models via Adversarial Noise Generation
Large-scale pre-trained vision-and-language (V+L) transformers have propelled the state of the art (SOTA) on Visual Question Answering (VQA) task. Despite impressive performance on the standard VQA benchmark, it remains …
Logical ReasoningQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)