The Future will be Different than Today: Model Evaluation Considerations when Developing Translational Clinical Biomarker
Finding translational biomarkers stands center stage of the future of personalized medicine in healthcare. We observed notable challenges in identifying robust biomarkers as some with great performance in one scenario often fail to perform well in new trials (e.g. different population, indications). With rapid development in the clinical trial world (e.g. assay, disease definition), new trials very likely differ from legacy ones in many perspectives and in development of biomarkers this heterogeneity should be considered. In response, we recommend considering building in the heterogeneity when evaluating biomarkers. In this paper, we present one evaluation strategy by using leave-one-study-out (LOSO) in place of conventional cross-validation (cv) methods to account for the potential heterogeneity across trials used for building and testing the biomarkers. To demonstrate the performance of K-fold vs LOSO cv in estimating the effect size of biomarkers, we leveraged data from clinical trials and simulation studies. In our assessment, LOSO cv provided a more objective estimate of the future performance. This conclusion remained true across different evaluation metrics and different statistical methods.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Uplifted Attackers, Human Defenders: The Cyber Offense-Defense Balance for Trailing-Edge Organizations
Advances in AI are widely understood to have implications for cybersecurity. Articles have emphasized the effect of AI on the cyber offense-defense balance, and commentators can be found arguing either that cyber will pr…
See Tomorrow, Act Today: Foresight-Driven Autonomous Driving
Current end-to-end autonomous driving planners are fundamentally reactive: they condition on historical and present observations to predict future actions. We argue that autonomous agents should instead imagine future sc…
Autonomous DrivingPlanning for net zero by 2050, what HVAC system interventions will today's code minimum commercial buildings require?
Heating, Ventilation and Air conditioning (HVAC) systems account for approximately 40% of the total energy used by buildings in the USA. To reduce this consumption States enforce minimum energy codes that currently range…
State Estimation for Future Energy Grids
Today's power generation and distribution networks are quickly moving toward automated control and integration of renewable resources - a complex, integrated system termed the Smart Grid. A key component in planning and …
State EstimationHigh Dynamic Range Imaging Technology
In this lecture note, we describe high dynamic range (HDR) imaging systems; such systems are able to represent luminances of much larger brightness and, typically, also a larger range of colors than conventional standard…
Vocal Bursts Intensity Prediction