Class Distribution Monitoring for Concept Drift Detection
We introduce Class Distribution Monitoring (CDM), an effective concept-drift detection scheme that monitors the class-conditional distributions of a datastream. In particular, our solution leverages multiple instances of an online and nonparametric change-detection algorithm based on QuantTree. CDM reports a concept drift after detecting a distribution change in any class, thus identifying which classes are affected by the concept drift. This can be precious information for diagnostics and adaptation. Our experiments on synthetic and real-world datastreams show that when the concept drift affects a few classes, CDM outperforms algorithms monitoring the overall data distribution, while achieving similar detection delays when the drift affects all the classes. Moreover, CDM outperforms comparable approaches that monitor the classification error, particularly when the change is not very apparent. Finally, we demonstrate that CDM inherits the properties of the underlying change detector, yielding an effective control over the expected time before a false alarm, or Average Run Length (ARL$_0$).
Code (1)
Tasks
Change DetectionDrift DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Beta Distribution Drift Detection for Adaptive Classifiers
With today's abundant streams of data, the only constant we can rely on is change. For stream classification algorithms, it is necessary to adapt to concept drift. This can be achieved by monitoring the model error, and …
Drift DetectionGeneral ClassificationA Neighbor-Searching Discrepancy-based Drift Detection Scheme for Learning Evolving Data
Uncertain changes in data streams present challenges for machine learning models to dynamically adapt and uphold performance in real-time. Particularly, classification boundary change, also known as real concept drift, i…
ClassificationDrift DetectionUnsupervised Concept Drift Detection from Deep Learning Representations in Real-time
Concept Drift is a phenomenon in which the underlying data distribution and statistical properties of a target domain change over time, leading to a degradation of the model's performance. Consequently, models deployed i…
Drift DetectionAdversarial Attacks for Drift Detection
Concept drift refers to the change of data distributions over time. While drift poses a challenge for learning models, requiring their continual adaption, it is also relevant in system monitoring to detect malfunctions, …
Drift DetectionConcept Drift Monitoring and Diagnostics of Supervised Learning Models via Score Vectors
Supervised learning models are one of the most fundamental classes of models. Viewing supervised learning from a probabilistic perspective, the set of training data to which the model is fitted is usually assumed to foll…
Diagnostic