Interpretable Anomaly Detection with DIFFI: Depth-based Isolation Forest Feature Importance
Anomaly Detection is an unsupervised learning task aimed at detecting anomalous behaviours with respect to historical data. In particular, multivariate Anomaly Detection has an important role in many applications thanks to the capability of summarizing the status of a complex system or observed phenomenon with a single indicator (typically called `Anomaly Score') and thanks to the unsupervised nature of the task that does not require human tagging. The Isolation Forest is one of the most commonly adopted algorithms in the field of Anomaly Detection, due to its proven effectiveness and low computational complexity. A major problem affecting Isolation Forest is represented by the lack of interpretability, an effect of the inherent randomness governing the splits performed by the Isolation Trees, the building blocks of the Isolation Forest. In this paper we propose effective, yet computationally inexpensive, methods to define feature importance scores at both global and local level for the Isolation Forest. Moreover, we define a procedure to perform unsupervised feature selection for Anomaly Detection problems based on our interpretability method; such procedure also serves the purpose of tackling the challenging task of feature importance evaluation in unsupervised anomaly detection. We assess the performance on several synthetic and real-world datasets, including comparisons against state-of-the-art interpretability techniques, and make the code publicly available to enhance reproducibility and foster research in the field.
Code (1)
Tasks
Anomaly DetectionFeature Importancefeature selectionUnsupervised Anomaly DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Towards Explainable Anomaly Detection in Shared Mobility Systems
Shared mobility systems, such as bike-sharing networks, play a crucial role in urban transportation. Identifying anomalies in these systems is essential for optimizing operations, improving service reliability, and enhan…
Unsupervised Anomaly DetectionFeature ImportanceIsolation Mondrian Forest for Batch and Online Anomaly Detection
We propose a new method, named isolation Mondrian forest (iMondrian forest), for batch and online anomaly detection. The proposed method is a novel hybrid of isolation forest and Mondrian forest which are existing method…
Anomaly DetectionEnsemble LearningOut-of-Distribution DetectionExplainable Anomaly Detection for Electric Vehicles Charging Stations
Electric vehicles (EV) charging stations are one of the critical infrastructures needed to support the transition to renewable-energy-based mobility, but ensuring their reliability and efficiency requires effective anoma…
Unsupervised Anomaly DetectionFeature ImportanceIsolation Forest in Novelty Detection Scenario
Data mining offers a diverse toolbox for extracting meaningful structures from complex datasets, with anomaly detection emerging as a critical subfield particularly in the context of streaming or real-time data. Within a…
Anomaly DetectionNovelty DetectionIsolation-based Spherical Ensemble Representations for Anomaly Detection
Anomaly detection is a critical task in data mining and management with applications spanning fraud detection, network security, and log monitoring. Despite extensive research, existing unsupervised anomaly detection met…
Unsupervised Anomaly DetectionFraud Detection