A Latent Space Correlation-Aware Autoencoder for Anomaly Detection in Skewed Data
Unsupervised learning-based anomaly detection in latent space has gained importance since discriminating anomalies from normal data becomes difficult in high-dimensional space. Both density estimation and distance-based methods to detect anomalies in latent space have been explored in the past. These methods prove that retaining valuable properties of input data in latent space helps in the better reconstruction of test data. Moreover, real-world sensor data is skewed and non-Gaussian in nature, making mean-based estimators unreliable for skewed data. Again, anomaly detection methods based on reconstruction error rely on Euclidean distance, which does not consider useful correlation information in the feature space and also fails to accurately reconstruct the data when it deviates from the training distribution. In this work, we address the limitations of reconstruction error-based autoencoders and propose a kernelized autoencoder that leverages a robust form of Mahalanobis distance (MD) to measure latent dimension correlation to effectively detect both near and far anomalies. This hybrid loss is aided by the principle of maximizing the mutual information gain between the latent dimension and the high-dimensional prior data space by maximizing the entropy of the latent space while preserving useful correlation information of the original data in the low-dimensional latent space. The multi-objective function has two goals -- it measures correlation information in the latent feature space in the form of robust MD distance and simultaneously tries to preserve useful correlation information from the original data space in the latent space by maximizing mutual information between the prior and latent space.
Code (0)
등록된 구현이 없습니다.
Tasks
Anomaly DetectionDensity EstimationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Enhancing anomaly detection with topology-aware autoencoders
Anomaly detection in high-energy physics is essential for identifying new physics beyond the Standard Model. Autoencoders provide a signal-agnostic approach but are limited by the topology of their latent space. This wor…
Anomaly DetectionTrustworthy Intrusion Detection: Confidence Estimation Using Latent Space
This work introduces a novel method for enhancing confidence in anomaly detection in Intrusion Detection Systems (IDS) through the use of a Variational Autoencoder (VAE) architecture. By developing a confidence metric de…
Anomaly DetectionBinary ClassificationIntrusion DetectionCorrelation-aware Deep Generative Model for Unsupervised Anomaly Detection
Unsupervised anomaly detection aims to identify anomalous samples from highly complex and unstructured data, which is pervasive in both fundamental research and industrial applications. However, most existing methods neg…
Anomaly DetectionDecoderRepresentation LearningUnsupervised Anomaly DetectionAnomaly Detection with Adversarially Learned Perturbations of Latent Space
Anomaly detection is to identify samples that do not conform to the distribution of the normal data. Due to the unavailability of anomalous data, training a supervised deep neural network is a cumbersome task. As such, u…
Anomaly DetectionUnsupervised Anomaly DetectionImproved Anomaly Detection through Conditional Latent Space VAE Ensembles
We propose a novel Conditional Latent space Variational Autoencoder (CL-VAE) to perform improved pre-processing for anomaly detection on data with known inlier classes and unknown outlier classes. This proposed variation…
Anomaly Detection