Developing a Dataset-Adaptive, Normalized Metric for Machine Learning Model Assessment: Integrating Size, Complexity, and Class Imbalance
Traditional metrics like accuracy, F1-score, and precision are frequently used to evaluate machine learning models, however they may not be sufficient for evaluating performance on tiny, unbalanced, or high-dimensional datasets. A dataset-adaptive, normalized metric that incorporates dataset characteristics like size, feature dimensionality, class imbalance, and signal-to-noise ratio is presented in this study. Early insights into the model's performance potential in challenging circumstances are provided by the suggested metric, which offers a scalable and adaptable evaluation framework. The metric's capacity to accurately forecast model scalability and performance is demonstrated via experimental validation spanning classification, regression, and clustering tasks, guaranteeing solid assessments in settings with limited data. This method has important ramifications for effective resource allocation and model optimization in machine learning workflows.
Code (0)
등록된 구현이 없습니다.
Tasks
Model OptimizationSimilar Papers 제목 키워드 기반
Adaptive and non-adaptive minimax rates for weighted Laplacian-eigenmap based nonparametric regression
We show both adaptive and non-adaptive minimax rates of convergence for a family of weighted Laplacian-Eigenmap based nonparametric regression methods, when the true regression function belongs to a Sobolev space and the…
regressionExploring Database Normalization Effects on SQL Generation
Schema design, particularly normalization, is a critical yet often overlooked factor in natural language to SQL (NL2SQL) systems. Most prior research evaluates models on fixed schemas, overlooking the influence of design…
Type predictionApplying Machine Learning Algorithms for Kidney Disease Diagnosis
Kidney disease is a ‘silent killer’; it usually develops over time, and many people do not know they have it until it is very far along. Chronic Kidney Disease (represents a major public health problem in both developed …
BIG-bench Machine LearningOptimization on Product Submanifolds of Convolution Kernels
Recent advances in optimization methods used for training convolutional neural networks (CNNs) with kernels, which are normalized according to particular constraints, have shown remarkable success. This work introduces a…
Developing and Validating Semi-Markov Occupancy Generative Models: A Technical Report
This report documents recent technical work on developing and validating stochastic occupancy models in commercial buildings, performed by the Pacific Northwest National Laboratory (PNNL) as part of the Sensor Impact Eva…