Anomaly Detection for High-Dimensional Data Using Large Deviations Principle
Most current anomaly detection methods suffer from the curse of dimensionality when dealing with high-dimensional data. We propose an anomaly detection algorithm that can scale to high-dimensional data using concepts from the theory of large deviations. The proposed Large Deviations Anomaly Detection (LAD) algorithm is shown to outperform state of art anomaly detection methods on a variety of large and high-dimensional benchmark data sets. Exploiting the ability of the algorithm to scale to high-dimensional data, we propose an online anomaly detection method to identify anomalies in a collection of multivariate time series. We demonstrate the applicability of the online algorithm in identifying counties in the United States with anomalous trends in terms of COVID-19 related cases and deaths. Several of the identified anomalous counties correlate with counties with documented poor response to the COVID pandemic.
Code (0)
등록된 구현이 없습니다.
Tasks
Anomaly DetectionTime SeriesTime Series AnalysisVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
A Flexible Framework for Anomaly Detection via Dimensionality Reduction
Anomaly detection is challenging, especially for large datasets in high dimensions. Here we explore a general anomaly detection framework based on dimensionality reduction and unsupervised clustering. We release DRAMA, a…
Active LearningAnomaly DetectionClusteringDimensionality ReductionText Anomaly Detection with Simplified Isolation Kernel
Two-step approaches combining pre-trained large language model embeddings and anomaly detectors demonstrate strong performance in text anomaly detection by leveraging rich semantic representations. However, high-dimensio…
Computational EfficiencyAnomaly DetectionSub-clusters of Normal Data for Anomaly Detection
Anomaly detection in data analysis is an interesting but still challenging research topic in real world applications. As the complexity of data dimension increases, it requires to understand the semantic contexts in its …
Anomaly DetectionClusteringClustered Hierarchical Anomaly and Outlier Detection Algorithms
Anomaly and outlier detection is a long-standing problem in machine learning. In some cases, anomaly detection is easy, such as when data are drawn from well-characterized distributions such as the Gaussian. However, whe…
Anomaly DetectionClusteringOutlier DetectionTransfer Learning+1Network Anomaly Detection for IoT Using Hyperdimensional Computing on NSL-KDD
With the rapid growth of IoT devices, ensuring robust network security has become a critical challenge. Traditional intrusion detection systems (IDSs) often face limitations in detecting sophisticated attacks within high…
Anomaly DetectionIntrusion Detection