TedEval: A Fair Evaluation Metric for Scene Text Detectors
Despite the recent success of scene text detection methods, common evaluation metrics fail to provide a fair and reliable comparison among detectors. They have obvious drawbacks in reflecting the inherent characteristic of text detection tasks, unable to address issues such as granularity, multiline, and character incompleteness. In this paper, we propose a novel evaluation protocol called TedEval (Text detector Evaluation), which evaluates text detections by an instance-level matching and a character-level scoring. Based on a firm standard rewarding behaviors that result in successful recognition, TedEval can act as a reliable standard for comparing and quantizing the detection quality throughout all difficulty levels. In this regard, we believe that TedEval can play a key role in developing state-of-the-art scene text detectors. The code is publicly available at https://github.com/clovaai/TedEval.
Code (1)
Tasks
Scene Text DetectionText DetectionSimilar Papers 제목 키워드 기반
Design and Evaluation of Deep Learning-Based Dual-Spectrum Image Fusion Methods
Visible images offer rich texture details, while infrared images emphasize salient targets. Fusing these complementary modalities enhances scene understanding, particularly for advanced vision tasks under challenging con…
FairnessScene UnderstandingWhat Makes a Scene ? Scene Graph-based Evaluation and Feedback for Controllable Generation
While text-to-image generation has been extensively studied, generating images from scene graphs remains relatively underexplored, primarily due to challenges in accurately modeling spatial relationships and object inter…
Image GenerationScene GenerationText to Image GenerationText-to-Image GenerationFairness and underspecification in acoustic scene classification: The case for disaggregated evaluations
Underspecification and fairness in machine learning (ML) applications have recently become two prominent issues in the ML community. Acoustic scene classification (ASC) applications have so far remained unaffected by thi…
Acoustic Scene ClassificationFairnessScene ClassificationCoTK: An Open-Source Toolkit for Fast Development and Fair Evaluation of Text Generation
In text generation evaluation, many practical issues, such as inconsistent experimental settings and metric implementations, are often ignored but lead to unfair evaluation and untenable conclusions. We present CoTK, an …
Text GenerationOn the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations
Multiple metrics have been introduced to measure fairness in various natural language processing tasks. These metrics can be roughly categorized into two categories: 1) \emph{extrinsic metrics} for evaluating fairness in…
Fairness