Autoencoder-based Anomaly Detection in Streaming Data with Incremental Learning and Concept Drift Adaptation
In our digital universe nowadays, enormous amount of data are produced in a streaming manner in a variety of application areas. These data are often unlabelled. In this case, identifying infrequent events, such as anomalies, poses a great challenge. This problem becomes even more difficult in non-stationary environments, which can cause deterioration of the predictive performance of a model. To address the above challenges, the paper proposes an autoencoder-based incremental learning method with drift detection (strAEm++DD). Our proposed method strAEm++DD leverages on the advantages of both incremental learning and drift detection. We conduct an experimental study using real-world and synthetic datasets with severe or extreme class imbalance, and provide an empirical analysis of strAEm++DD. We further conduct a comparative study, showing that the proposed method significantly outperforms existing baseline and advanced methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Anomaly DetectionDrift DetectionIncremental LearningSimilar Papers 제목 키워드 기반
Deep Autoencoders with Value-at-Risk Thresholding for Unsupervised Anomaly Detection
Many real-world monitoring and surveillance applications require non-trivial anomaly detection to be run in the streaming model. We consider an incremental-learning approach, wherein a deep-autoencoding (DAE) model of wh…
Anomaly DetectionIncremental Learningsubspace methodsUnsupervised Anomaly DetectionInQMAD: Incremental Quantum Measurement Anomaly Detection
Streaming anomaly detection refers to the problem of detecting anomalous data samples in streams of data. This problem poses challenges that classical and deep anomaly detection methods are not designed to cope with, suc…
Anomaly DetectionDensity EstimationAdaptive Model Pooling for Online Deep Anomaly Detection from a Complex Evolving Data Stream
Online anomaly detection from a data stream is critical for the safety and security of many applications but is facing severe challenges due to complex and evolving data streams from IoT devices and cloud-based infrastru…
Anomaly DetectionAbuse and Fraud Detection in Streaming Services Using Heuristic-Aware Machine Learning
This work presents a fraud and abuse detection framework for streaming services by modeling user streaming behavior. The goal is to discover anomalous and suspicious incidents and scale the investigation efforts by creat…
Abuse DetectionAnomaly DetectionBIG-bench Machine LearningBinary Classification+7Drift-Aware Variational Autoencoder-based Anomaly Detection with Two-level Ensembling
In today's digital world, the generation of vast amounts of streaming data in various domains has become ubiquitous. However, many of these data are unlabeled, making it challenging to identify events, particularly anoma…
Incremental LearningAnomaly Detection