Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matching
Formula recognition presents significant challenges due to the complicated structure and varied notation of mathematical expressions. Despite continuous advancements in formula recognition models, the evaluation metrics employed by these models, such as BLEU and Edit Distance, still exhibit notable limitations. They overlook the fact that the same formula has diverse representations and is highly sensitive to the distribution of training data, thereby causing unfairness in formula recognition evaluation. To this end, we propose a Character Detection Matching (CDM) metric, ensuring the evaluation objectivity by designing an image-level rather than a LaTeX-level metric score. Specifically, CDM renders both the model-predicted LaTeX and the ground-truth LaTeX formulas into image-formatted formulas, then employs visual feature extraction and localization techniques for precise character-level matching, incorporating spatial position information. Such a spatially-aware and character-matching method offers a more accurate and equitable evaluation compared with previous BLEU and Edit Distance metrics that rely solely on text-based character matching. Experimentally, we evaluated various formula recognition models using CDM, BLEU, and ExpRate metrics. Their results demonstrate that the CDM aligns more closely with human evaluation standards and provides a fairer comparison across different models by eliminating discrepancies caused by diverse formula representations. Code is available at https://github.com/opendatalab/UniMERNet/tree/main/cdm.
Code (1)
Similar Papers 제목 키워드 기반
Applications of Generative Adversarial Models in Visual Search Reformulation
Query reformulation is the process by which a input search query is refined by the user to match documents outside the original top-n results. On average, roughly 50% of text search queries involve some form of reformula…
Capability-aware Prompt Reformulation Learning for Text-to-Image Generation
Text-to-image generation systems have emerged as revolutionary tools in the realm of artistic creation, offering unprecedented ease in transforming textual prompts into visual art. However, the efficacy of these systems …
Image GenerationText to Image GenerationText-to-Image GenerationVisual Prompting in LLMs for Enhancing Emotion Recognition
Vision Large Language Models (VLLMs) are transforming the intersection of computer vision and natural language processing. Nonetheless, the potential of using visual prompts for emotion recognition in these models remain…
Emotion RecognitionVisual PromptingAltitude-Adaptive Vision-Only Geo-Localization for UAVs in GPS-Denied Environments
To address the scale mismatch caused by large altitude variations in UAV visual place recognition, we propose a monocular vision-only altitude-adaptive geo-localization framework. The method first estimates relative alti…
Visual Place RecognitionImage-to-Graph Transformers for Chemical Structure Recognition
For several decades, chemical knowledge has been published in written text, and there have been many attempts to make it accessible, for example, by transforming such natural language text to a structured format. Althoug…