On the Change of Decision Boundaries and Loss in Learning with Concept Drift
The notion of concept drift refers to the phenomenon that the distribution generating the observed data changes over time. If drift is present, machine learning models may become inaccurate and need adjustment. Many technologies for learning with drift rely on the interleaved test-train error (ITTE) as a quantity which approximates the model generalization error and triggers drift detection and model updates. In this work, we investigate in how far this procedure is mathematically justified. More precisely, we relate a change of the ITTE to the presence of real drift, i.e., a changed posterior, and to a change of the training result under the assumption of optimality. We support our theoretical findings by empirical evidence for several learning algorithms, models, and datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Drift DetectionSimilar Papers 제목 키워드 기반
Concept Drift Detection and Adaptation with Weak Supervision on Streaming Unlabeled Data
Concept drift in learning and classification occurs when the statistical properties of either the data features or target change over time; evidence of drift has appeared in search data, medical research, malware, web da…
ClusteringDrift DetectionTackling Virtual and Real Concept Drifts: An Adaptive Gaussian Mixture Model
Real-world applications have been dealing with large amounts of data that arrive over time and generally present changes in their underlying joint probability distribution, i.e., concept drift. Concept drift can be subdi…
CONDA-PM -- A Systematic Review and Framework for Concept Drift Analysis in Process Mining
Business processes evolve over time to adapt to changing business environments. This requires continuous monitoring of business processes to gain insights into whether they conform to the intended design or deviate from …
ManagementSystematic Literature ReviewThe Utility of Hyperplane Angle Metric in Detecting Financial Concept Drift
In financial time series analysis, introducing a new metric for concept drift is essential to address the limitations of existing evaluation methods, particularly in terms of speed, interpretability and stability. Perfor…
ARCDrift DetectionGeometric MatchingTime Series+1Explaining Concept Drift through the Evolution of Group Counterfactuals
Machine learning models in dynamic environments often suffer from concept drift, where changes in the data distribution degrade performance. While detecting this drift is a well-studied topic, explaining how and why the …