Inference for ROC Curves Based on Estimated Predictive Indices
We provide a comprehensive theory of conducting in-sample statistical inference about receiver operating characteristic (ROC) curves that are based on predicted values from a first stage model with estimated parameters (such as a logit regression). The term "in-sample" refers to the practice of using the same data for model estimation (training) and subsequent evaluation, i.e., the construction of the ROC curve. We show that in this case the first stage estimation error has a generally non-negligible impact on the asymptotic distribution of the ROC curve and develop the appropriate pointwise and functional limit theory. We propose methods for simulating the distribution of the limit process and show how to use the results in practice in comparing ROC curves.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Learning Rich Geographical Representations: Predicting Colorectal Cancer Survival in the State of Iowa
Neural networks are capable of learning rich, nonlinear feature representations shown to be beneficial in many predictive tasks. In this work, we use these models to explore the use of geographical features in predicting…
ClusteringFactor Importance Ranking and Selection using Total Indices
Factor importance measures the impact of each feature on output prediction accuracy. Many existing works focus on the model-based importance, but an important feature in one learning algorithm may hold little significanc…
Binary ClassificationFunctional Factor Regression with an Application to Electricity Price Curve Modeling
We propose a function-on-function linear regression model for time-dependent curve data that is consistently estimated by imposing factor structures on the regressors. An integral operator based on cross-covariances iden…
regressionvalidInferring the dynamics of oscillatory systems using recurrent neural networks
We investigate the predictive power of recurrent neural networks for oscillatory systems not only on the attractor, but in its vicinity as well. For this we consider systems perturbed by an external force. This allows us…
Synthetic Survival Data Generation for Heart Failure Prognosis Using Deep Generative Models
Background: Heart failure (HF) research is constrained by limited access to large, shareable datasets due to privacy regulations and institutional barriers. Synthetic data generation offers a promising solution to overco…
Synthetic Data Generation