Suitability of Different Metric Choices for Concept Drift Detection
The notion of concept drift refers to the phenomenon that the distribution, which is underlying the observed data, changes over time; as a consequence machine learning models may become inaccurate and need adjustment. Many unsupervised approaches for drift detection rely on measuring the discrepancy between the sample distributions of two time windows. This may be done directly, after some preprocessing (feature extraction, embedding into a latent space, etc.), or with respect to inferred features (mean, variance, conditional probabilities etc.). Most drift detection methods can be distinguished in what metric they use, how this metric is estimated, and how the decision threshold is found. In this paper, we analyze structural properties of the drift induced signals in the context of different metrics. We compare different types of estimators and metrics theoretically and empirically and investigate the relevance of the single metric components. In addition, we propose new choices and demonstrate their suitability in several experiments.
Code (0)
등록된 구현이 없습니다.
Tasks
Drift DetectionSimilar Papers 제목 키워드 기반
One or Two Things We know about Concept Drift -- A Survey on Monitoring Evolving Environments
The world surrounding us is subject to constant change. These changes, frequently described as concept drift, influence many industrial and technical processes. As they can lead to malfunctions and other anomalous behavi…
Anomaly DetectionDrift DetectionSystematic Literature ReviewAre Concept Drift Detectors Reliable Alarming Systems? -- A Comparative Study
As machine learning models increasingly replace traditional business logic in the production system, their lifecycle management is becoming a significant concern. Once deployed into production, the machine learning model…
Drift DetectionManagementInvestigating the Suitability of Concept Drift Detection for Detecting Leakages in Water Distribution Networks
Leakages are a major risk in water distribution networks as they cause water loss and increase contamination risks. Leakage detection is a difficult task due to the complex dynamics of water distribution networks. In par…
Drift DetectionTackling Virtual and Real Concept Drifts: An Adaptive Gaussian Mixture Model
Real-world applications have been dealing with large amounts of data that arrive over time and generally present changes in their underlying joint probability distribution, i.e., concept drift. Concept drift can be subdi…
Detecting Concept Drift in the Presence of Sparsity -- A Case Study of Automated Change Risk Assessment System
Missing values, widely called as \textit{sparsity} in literature, is a common characteristic of many real-world datasets. Many imputation methods have been proposed to address this problem of data incompleteness or spars…
Drift DetectionImputationMissing Values