Unsupervised Training Data Generation of Handwritten Formulas using Generative Adversarial Networks with Self-Attention
The recognition of handwritten mathematical expressions in images and video frames is a difficult and unsolved problem yet. Deep convectional neural networks are basically a promising approach, but typically require a large amount of labeled training data. However, such a large training dataset does not exist for the task of handwritten formula recognition. In this paper, we introduce a system that creates a large set of synthesized training examples of mathematical expressions which are derived from LaTeX documents. For this purpose, we propose a novel attention-based generative adversarial network to translate rendered equations to handwritten formulas. The datasets generated by this approach contain hundreds of thousands of formulas, making it ideal for pretraining or the design of more complex models. We evaluate our synthesized dataset and the recognition approach on the CROHME 2014 benchmark dataset. Experimental results demonstrate the feasibility of the approach.
Code (1)
Tasks
Generative Adversarial NetworkSimilar Papers 제목 키워드 기반
Image-to-LaTeX Converter for Mathematical Formulas and Text
In this project, we train a vision encoder-decoder model to generate LaTeX code from images of mathematical formulas and text. Utilizing a diverse collection of image-to-LaTeX data, we build two models: a base model with…
DecoderGPUDatabase of handwritten Arabic mathematical formulas images
Although publicly available, ground-truthed database have proven useful for training, evaluating, and comparing recognition systems in many domains, the availability of such database for handwritten Arabic mathematical f…
Learning Heuristics for Quantified Boolean Formulas through Reinforcement Learning
We demonstrate how to learn efficient heuristics for automated reasoning algorithms for quantified Boolean formulas through deep reinforcement learning. We focus on a backtracking search algorithm, which can already solv…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Learning Heuristics for Quantified Boolean Formulas through Deep Reinforcement Learning
We demonstrate how to learn efficient heuristics for automated reasoning algorithms for quantified Boolean formulas through deep reinforcement learning. We focus on a backtracking search algorithm, which can already solv…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)WildHandBench: A Benchmark for Handwritten Text Understanding that Challenges MLLMs and Humans
While the top model on OmniDocBench now reaches 96.34% overall on printed-document parsing, the ability of current models to handle challenging handwritten documents remains largely uncharacterized. Existing benchmarks f…