A GRU-based Encoder-Decoder Approach with Attention for Online Handwritten Mathematical Expression Recognition
In this study, we present a novel end-to-end approach based on the encoder-decoder framework with the attention mechanism for online handwritten mathematical expression recognition (OHMER). First, the input two-dimensional ink trajectory information of handwritten expression is encoded via the gated recurrent unit based recurrent neural network (GRU-RNN). Then the decoder is also implemented by the GRU-RNN with a coverage-based attention model. The proposed approach can simultaneously accomplish the symbol recognition and structural analysis to output a character sequence in LaTeX format. Validated on the CROHME 2014 competition task, our approach significantly outperforms the state-of-the-art with an expression recognition accuracy of 52.43% by only using the official training dataset. Furthermore, the alignments between the input trajectories of handwritten expressions and the output LaTeX sequences are visualized by the attention mechanism to show the effectiveness of the proposed method.
Code (1)
Tasks
DecoderSimilar Papers 제목 키워드 기반
Stroke Constrained Attention Network for Online Handwritten Mathematical Expression Recognition
In this paper, we propose a novel stroke constrained attention network (SCAN) which treats stroke as the basic unit for encoder-decoder based online handwritten mathematical expression recognition (HMER). Unlike previous…
DecoderOffline Handwritten Mathematical Recognition using Adversarial Learning and Transformers
Offline Handwritten Mathematical Expression Recognition (HMER) is a major area in the field of mathematical expression recognition. Offline HMER is often viewed as a much harder problem as compared to online HMER due to …
DecoderMulti-Scale Attention with Dense Encoder for Handwritten Mathematical Expression Recognition
Handwritten mathematical expression recognition is a challenging problem due to the complicated two-dimensional structures, ambiguous handwriting input and variant scales of handwritten math symbols. To settle this probl…
DecoderHandwritten Mathmatical Expression RecognitionMathImproving Attention-Based Handwritten Mathematical Expression Recognition with Scale Augmentation and Drop Attention
Handwritten mathematical expression recognition (HMER) is an important research direction in handwriting recognition. The performance of HMER suffers from the two-dimensional structure of mathematical expressions (MEs). …
DecoderHandwriting RecognitionRobust Encoder-Decoder Learning Framework towards Offline Handwritten Mathematical Expression Recognition Based on Multi-Scale Deep Neural Network
Offline handwritten mathematical expression recognition is a challenging task, because handwritten mathematical expressions mainly have two problems in the process of recognition. On one hand, it is how to correctly reco…
Decoder