The Impact of Discretization Method on the Detection of Six Types of Anomalies in Datasets
Anomaly detection is the process of identifying cases, or groups of cases, that are in some way unusual and do not fit the general patterns present in the dataset. Numerous algorithms use discretization of numerical data in their detection processes. This study investigates the effect of the discretization method on the unsupervised detection of each of the six anomaly types acknowledged in a recent typology of data anomalies. To this end, experiments are conducted with various datasets and SECODA, a general-purpose algorithm for unsupervised non-parametric anomaly detection in datasets with numerical and categorical attributes. This algorithm employs discretization of continuous attributes, exponentially increasing weights and discretization cut points, and a pruning heuristic to detect anomalies with an optimal number of iterations. The results demonstrate that standard SECODA can detect all six types, but that different discretization methods favor the discovery of certain anomaly types. The main findings also hold for other detection techniques using discretization.
Code (1)
Tasks
Anomaly DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SECODA: Segmentation- and Combination-Based Detection of Anomalies
This study introduces SECODA, a novel general-purpose unsupervised non-parametric anomaly detection algorithm for datasets containing continuous and categorical attributes. The method is guaranteed to identify cases with…
Anomaly DetectionAttributeDetection of Thin Boundaries between Different Types of Anomalies in Outlier Detection using Enhanced Neural Networks
Outlier detection has received special attention in various fields, mainly for those dealing with machine learning and artificial intelligence. As strong outliers, anomalies are divided into the point, contextual and col…
Anomaly DetectionOutlier DetectionWhat Information Contributes to Log-based Anomaly Detection? Insights from a Configurable Transformer-Based Approach
Log data are generated from logging statements in the source code, providing insights into the execution processes of software applications and systems. State-of-the-art log-based anomaly detection approaches typically l…
Anomaly DetectionA Typology of Data Anomalies
Anomalies are cases that are in some way unusual and do not appear to fit the general patterns present in the dataset. Several conceptualizations exist to distinguish between different types of anomalies. However, these …
Anomaly DetectionConditioning Latent-Space Clusters for Real-World Anomaly Classification
Anomalies in the domain of autonomous driving are a major hindrance to the large-scale deployment of autonomous vehicles. In this work, we focus on high-resolution camera data from urban scenes that include anomalies of …
Anomaly ClassificationAutonomous DrivingAutonomous VehiclesClassification