EDSL: An Encoder-Decoder Architecture with Symbol-Level Features for Printed Mathematical Expression Recognition

2020-07-06 · Yingnan Fu, Tingting Liu, Ming Gao, Aoying Zhou

Printed Mathematical expression recognition (PMER) aims to transcribe a printed mathematical expression image into a structural expression, such as LaTeX expression. It is a crucial task for many applications, including automatic question recommendation, automatic problem solving and analysis of the students, etc. Currently, the mainstream solutions rely on solving image captioning tasks, all addressing image summarization. As such, these methods can be suboptimal for solving MER problem. In this paper, we propose a new method named EDSL, shorted for encoder-decoder with symbol-level features, to identify the printed mathematical expressions from images. The symbol-level image encoder of EDSL consists of segmentation module and reconstruction module. By performing segmentation module, we identify all the symbols and their spatial information from images in an unsupervised manner. We then design a novel reconstruction module to recover the symbol dependencies after symbol segmentation. Especially, we employ a position correction attention mechanism to capture the spatial relationships between symbols. To alleviate the negative impact from long output, we apply the transformer model for transcribing the encoded image into the sequential and structural output. We conduct extensive experiments on two real datasets to verify the effectiveness and rationality of our proposed EDSL method. The experimental results have illustrated that EDSL has achieved 92.7\% and 89.0\% in evaluation metric Match, which are 3.47\% and 4.04\% higher than the state-of-the-art method. Our code and datasets are available at https://github.com/abcAnonymous/EDSL .

📄 PDF Abstract BibTeX arXiv:2007.02517

Code (1)

abcAnonymous/EDSL 공식 구현 pytorch

Tasks

Decoder Image Captioning Segmentation

Compilation as a Typed EDSL-to-EDSL Transformation

2016-03-29 · Emil Axelsson

This article is about an implementation and compilation technique that is used in RAW-Feldspar which is a complete rewrite of the Feldspar embedded domain-specific language (EDSL) (Axelsson et al. 2010). Feldspar is high…

C++ code Translation

Federated Split Learning for Resource-Constrained Robots in Industrial IoT: Framework Comparison, Optimization Strategies, and Future Directions

2025-10-07 · Wanli Ni, Hui Tian, Shuai Wang, Chengyang Li 외 arxiv

Federated split learning (FedSL) has emerged as a promising paradigm for enabling collaborative intelligence in industrial Internet of Things (IoT) systems, particularly in smart factories where data privacy, communicati…

Model Compression

MeDSLIP: Medical Dual-Stream Language-Image Pre-training for Fine-grained Alignment

2024-03-15 · Wenrui Fan, Mohammod Naimul Islam Suvon, Shuo Zhou, Xianyuan Liu 외

Vision-language pre-training (VLP) models have shown significant advancements in the medical domain. Yet, most VLP models align raw reports to images at a very coarse level, without modeling fine-grained relationships be…

Anatomy Contrastive Learning Representation Learning

Graph-based Neural Modules to Inspect Attention-based Architectures: A Position Paper

2022-10-13 · Breno W. Carvalho, Artur D'Avilla Garcez, Luis C. Lamb

Encoder-decoder architectures are prominent building blocks of state-of-the-art solutions for tasks across multiple fields where deep learning (DL) or foundation models play a key role. Although there is a growing commun…

Decoder Position

FedSlate:A Federated Deep Reinforcement Learning Recommender System

2024-09-23 · Yongxin Deng, Xihe Qiu, Xiaoyu Tan, Yaochu Jin

Reinforcement learning methods have been used to optimize long-term user engagement in recommendation systems. However, existing reinforcement learning-based recommendation systems do not fully exploit the relevance of i…

Deep Reinforcement Learning Federated Learning Recommendation Systems reinforcement-learning+1