Interpretation of the Area Under the ROC Curve for Risk Prediction Models
The area under the curve (AUC) of the receiver operating characteristics curve (ROC) evaluates the separation between patients and nonpatients or discrimination. For risk prediction models these risk distributions can be derived from the population risk distribution so are not independent as in diagnosis. A ROC curve AUC formula based on the underlying population risk distribution clarifies how discrimination is defined mathematically and that generation of the equivalent c-statistic effects a Monte Carlo integration of the formula. For a selection of continuous risk distributions, exact analytic formulas or numerical results for the ROC curve AUC and overlap measure are presented and demonstrate a linear or near-linear dependence on their standard deviation. The ROC curve AUC is also shown to be highly dependent on the mean population risk, a distinction from the independence from disease prevalence for diagnostic tests. The converse of discrimination, overlap, has been quantified by the overlap measure, which appears to provide equivalent information. As achieving wider population risk distributions is the goal of risk prediction modeling for clinical risk stratification, interpreting the ROC curve AUC as a measure of dispersion, rather than discrimination, when comparing risk prediction models may be more relevant.
Code (0)
등록된 구현이 없습니다.
Tasks
DiagnosticPredictionSimilar Papers 제목 키워드 기반
Deep ROC Analysis and AUC as Balanced Average Accuracy to Improve Model Selection, Understanding and Interpretation
Optimal performance is critical for decision-making tasks from medicine to autonomous driving, however common performance measures may be too general or too specific. For binary classifiers, diagnostic tests or prognosis…
Autonomous DrivingDecision MakingDiagnosticModel Selection+3Interpretable estimation of the risk of heart failure hospitalization from a 30-second electrocardiogram
Survival modeling in healthcare relies on explainable statistical models; yet, their underlying assumptions are often simplistic and, thus, unrealistic. Machine learning models can estimate more complex relationships and…
An explainable Transformer-based deep learning model for the prediction of incident heart failure
Predicting the incidence of complex chronic conditions such as heart failure is challenging. Deep learning models applied to rich electronic health records may improve prediction but remain unexplainable hampering their …
Deep LearningPredictionReceiver operating characteristic (ROC) movies, universal ROC (UROC) curves, and coefficient of predictive ability (CPA)
Throughout science and technology, receiver operating characteristic (ROC) curves and associated area under the curve (AUC) measures constitute powerful tools for assessing the predictive abilities of features, markers a…
Binary ClassificationThe Consequences of the Framing of Machine Learning Risk Prediction Models: Evaluation of Sepsis in General Wards
Objectives: To evaluate the consequences of the framing of machine learning risk prediction models. We evaluate how framing affects model performance and model learning in four different approaches previously applied in …
Missing ValuesPrediction