On Leakage in Machine Learning Pipelines
Machine learning (ML) provides powerful tools for predictive modeling. ML's popularity stems from the promise of sample-level prediction with applications across a variety of fields from physics and marketing to healthcare. However, if not properly implemented and evaluated, ML pipelines may contain leakage typically resulting in overoptimistic performance estimates and failure to generalize to new data. This can have severe negative financial and societal implications. Our aim is to expand understanding associated with causes leading to leakage when designing, implementing, and evaluating ML pipelines. Illustrated by concrete examples, we provide a comprehensive overview and discussion of various types of leakage that may arise in ML pipelines.
Code (0)
등록된 구현이 없습니다.
Tasks
MarketingSimilar Papers 제목 키워드 기반
bioLeak: Leakage-Aware Modeling and Diagnostics for Machine Learning in R
Data leakage remains a recurrent source of optimistic bias in biomedical machine learning studies. Standard row-wise cross-validation and globally estimated preprocessing steps are often inappropriate for data with repea…
Binary ClassificationImpact of Leakage on Data Harmonization in Machine Learning Pipelines in Class Imbalance Across Sites
Machine learning (ML) models benefit from large datasets. Collecting data in biomedical domains is costly and challenging, hence, combining datasets has become a common practice. However, datasets obtained under differen…
A Simple Probabilistic Model With Extended Kalman Filter To Predict Multi-leak In Pipelines
Pipelines for water supply are susceptible to burst-leakage due to fluid pressures of various nature. High pressure heads resulting in circumferential and (or) axial stresses larger than the material yield stress could c…
ERP-RiskBench: Leakage-Safe Ensemble Learning for Financial Risk
Financial risk detection in Enterprise Resource Planning (ERP) systems is an important but underexplored application of machine learning. Published studies in this area tend to suffer from vague dataset descriptions, lea…
Ensemble LearningHidden Markov Models for Pipeline Damage Detection Using Piezoelectric Transducers
Oil and gas pipeline leakages lead to not only enormous economic loss but also environmental disasters. How to detect the pipeline damages including leakages and cracks has attracted much research attention. One of the p…