ASI: Accuracy-Stability Index for Evaluating Deep Learning Models
In the context of deep learning research, where model introductions continually occur, the need for effective and efficient evaluation remains paramount. Existing methods often emphasize accuracy metrics, overlooking stability. To address this, the paper introduces the Accuracy-Stability Index (ASI), a quantitative measure incorporating both accuracy and stability for assessing deep learning models. Experimental results demonstrate the application of ASI, and a 3D surface model is presented for visualizing ASI, mean accuracy, and coefficient of variation. This paper addresses the important issue of quantitative benchmarking metrics for deep learning models, providing a new approach for accurately evaluating accuracy and stability of deep learning models. The paper concludes with discussions on potential weaknesses and outlines future research directions.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingDeep LearningSimilar Papers 제목 키워드 기반
Evaluating Large Language Models in Code Generation: INFINITE Methodology for Defining the Inference Index
This study introduces a new methodology for an Inference Index (InI), called INFerence INdex In Testing model Effectiveness methodology (INFINITE), aiming to evaluate the performance of Large Language Models (LLMs) in co…
Code GenerationFacets of Disparate Impact: Evaluating Legally Consistent Bias in Machine Learning
Leveraging current legal standards, we define bias through the lens of marginal benefits and objective testing with the novel metric "Objective Fairness Index". This index combines the contextual nuances of objective tes…
FairnessTrustworthiness Calibration Framework for Phishing Email Detection Using Large Language Models
Phishing emails continue to pose a persistent challenge to online communication, exploiting human trust and evading automated filters through realistic language and adaptive tactics. While large language models (LLMs) su…
Text ClassificationA Network Decoupling Method for Voltage Stability Analysis Based on Holomorphic Embedding
This paper proposes a network decoupling method based on Holomorphic Embedding (HE) for voltage stability analysis. Using the proposed HE method with a physical load scaling factor s, it develops a set of decoupled two-b…
Exploiting a Supervised Index for High-accuracy Parameter Estimation in Low SNR
Performance of parameter estimation is one of the most important issues in array signal processing. The root mean square error, probability of success, resolution probabilities, and computational complexity are frequentl…
parameter estimation