Semi-supervised anomaly detection algorithm based on KL divergence (SAD-KL)
The unlabeled data are generally assumed to be normal data in detecting abnormal data via semisupervised learning. This assumption, however, causes inevitable detection error when distribution of unlabeled data is different from distribution of labeled normal dataset. To deal the problem caused by distribution gap between labeled and unlabeled data, we propose a semi-supervised anomaly detection algorithm using KL divergence (SAD-KL). The proposed SAD-KL is composed of two steps: (1) estimating KL divergence of probability density functions (PDFs) of the local outlier factors (LOFs) of the labeled normal data and the unlabeled data (2) estimating detection probability and threshold for detecting normal data in unlabeled data by using the KL divergence. We show that the PDFs of the LOFs follow Burr distribution and use them for detection. Once the threshold is computed, the SAD-KL runs iteratively until the labeling change rate is lower than the predefined threshold. Experiments results show that the SAD-KL shows superior detection probability over the existing algorithms even though it takes less learning time.
Code (0)
등록된 구현이 없습니다.
Tasks
Anomaly DetectionSemi-supervised Anomaly DetectionSupervised Anomaly DetectionSimilar Papers 제목 키워드 기반
ESAD: End-to-end Deep Semi-supervised Anomaly Detection
This paper explores semi-supervised anomaly detection, a more practical setting for anomaly detection where a small additional set of labeled samples are provided. We propose a new KL-divergence based objective function …
Anomaly DetectionDecoderMedical DiagnosisSemi-supervised Anomaly Detection+1PAC-Wrap: Semi-Supervised PAC Anomaly Detection
Anomaly detection is essential for preventing hazardous outcomes for safety-critical applications like autonomous driving. Given their safety-criticality, these applications benefit from provable bounds on various errors…
Anomaly DetectionAutonomous DrivingUnsupervised Anomaly DetectionAnomaly Detection in Images
Visual defect assessment is a form of anomaly detection. This is very relevant in finding faults such as cracks and markings in various surface inspection tasks like pavement and automotive parts. The task involves detec…
Anomaly DetectionSupervised Anomaly DetectionTransfer LearningNNG-Mix: Improving Semi-supervised Anomaly Detection with Pseudo-anomaly Generation
Anomaly detection (AD) is essential in identifying rare and often critical events in complex systems, finding applications in fields such as network intrusion detection, financial fraud detection, and fault detection in …
Anomaly DetectionData AugmentationFault DetectionFraud Detection+4Understanding Bias in Anomaly Detection: A Semi-Supervised View with PAC Guarantees
Anomaly detection presents a unique challenge in machine learning, due to the scarcity of labeled anomaly data. Existing work attempts to mitigate such problems via semi-supervised learning, i.e., augmenting unsupervised…
Anomaly DetectionSemi-supervised Anomaly DetectionSupervised Anomaly DetectionUnsupervised Anomaly Detection