Automating concept-drift detection by self-evaluating predictive model degradation
A key aspect of automating predictive machine learning entails the capability of properly triggering the update of the trained model. To this aim, suitable automatic solutions to self-assess the prediction quality and the data distribution drift between the original training set and the new data have to be devised. In this paper, we propose a novel methodology to automatically detect prediction-quality degradation of machine learning models due to class-based concept drift, i.e., when new data contains samples that do not fit the set of class labels known by the currently-trained predictive model. Experiments on synthetic and real-world public datasets show the effectiveness of the proposed methodology in automatically detecting and describing concept drift caused by changes in the class-label data distributions.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningDrift DetectionSimilar Papers 제목 키워드 기반
How well does Classification Accuracy capture Concept Drift Detection Quality? An overview of Concept Drift Detection evaluation
Data streams are nowadays among the most frequently analyzed data structures, with the concept drift posing a major challenge encountered by processing systems. Despite the proposition of numerous solutions to counteract…
A Framework for Explainable Concept Drift Detection in Process Mining
Rapidly changing business environments expose companies to high levels of uncertainty. This uncertainty manifests itself in significant changes that tend to occur over the lifetime of a process and possibly affect its pe…
Drift DetectionIdentifying Predictions That Influence the Future: Detecting Performative Concept Drift in Data Streams
Concept Drift has been extensively studied within the context of Stream Learning. However, it is often assumed that the deployed model's predictions play no role in the concept drift the system experiences. Closer inspec…
Drift DetectionEmpirical Evaluation of Concept Drift in ML-Based Android Malware Detection
Despite outstanding results, machine learning-based Android malware detection models struggle with concept drift, where rapidly evolving malware characteristics degrade model effectiveness. This study examines the impact…
Few-Shot LearningMalware DetectionMORPH: Towards Automated Concept Drift Adaptation for Malware Detection
Concept drift is a significant challenge for malware detection, as the performance of trained machine learning models degrades over time, rendering them impractical. While prior research in malware concept drift adaptati…
Active LearningMalware DetectionMORPHPseudo Label