paper-with-me

홈 › Papers

Confidence intervals for the Cox model test error from cross-validation

2022-01-26 · Min Woo Sun, Robert Tibshirani

Cross-validation (CV) is one of the most widely used techniques in statistical learning for estimating the test error of a model, but its behavior is not yet fully understood. It has been shown that standard confidence intervals for test error using estimates from CV may have coverage below nominal levels. This phenomenon occurs because each sample is used in both the training and testing procedures during CV and as a result, the CV estimates of the errors become correlated. Without accounting for this correlation, the estimate of the variance is smaller than it should be. One way to mitigate this issue is by estimating the mean squared error of the prediction error instead using nested CV. This approach has been shown to achieve superior coverage compared to intervals derived from standard CV. In this work, we generalize the nested CV idea to the Cox proportional hazards model and explore various choices of test error for this setting.

📄 PDF Abstract BibTeX arXiv:2201.10770

Code (1)

minwoosun/nestedcv-confint 공식 구현

Similar Papers 제목 키워드 기반

Cross-validation Confidence Intervals for Test Error

2020-07-24 · NeurIPS 2020 12 · Pierre Bayle, Alexandre Bayle, Lucas Janson, Lester Mackey

This work develops central limit theorems for cross-validation and consistent estimators of its asymptotic variance under weak stability conditions on the learning algorithm. Together, these results provide practical, as…

valid

Cross-validation: what does it estimate and how well does it do it?

2021-04-01 · Stephen Bates, Trevor Hastie, Robert Tibshirani

Cross-validation is a widely-used technique to estimate prediction error, but its behavior is complex and not fully understood. Ideally, one would like to think that cross-validation estimates the prediction error for th…

Prediction

Biomechanical Reconstruction with Confidence Intervals from Multiview Markerless Motion Capture

2025-02-10 · R. James Cotton, Fabian Sinz

Advances in multiview markerless motion capture (MMMC) promise high-quality movement analysis for clinical practice and research. While prior validation studies show MMMC performs well on average, they do not provide wha…

Markerless Motion Capture

Please, Don't Forget the Difference and the Confidence Interval when Seeking for the State-of-the-Art Status

2022-05-23 · Yves Bestgen

This paper argues for the widest possible use of bootstrap confidence intervals for comparing NLP system performances instead of the state-of-the-art status (SOTA) and statistical significance testing. Their main benefit…

Confidence Calibration for Systems with Cascaded Predictive Modules

2023-09-21 · Yunye Gong, Yi Yao, Xiao Lin, Ajay Divakaran 외

Existing conformal prediction algorithms estimate prediction intervals at target confidence levels to characterize the performance of a regression model on new test samples. However, considering an autonomous system cons…

Conformal PredictionPredictionPrediction Intervalsregression