Unsupervised real-time anomaly detection for streaming data

2017-06-02 · Neurocomputing 2017 6 · Subutai Ahmad;Alexander Lavin;Scott Purdy; Zuha Agha

We are seeing an enormous increase in the availability of streaming, time-series data. Largely driven by the rise of connected real-time data sources, this data presents technical challenges and opportunities. One fundamental capability for streaming analytics is to model each stream in an unsupervised fashion and detect unusual, anomalous behaviors in real-time. Early anomaly detection is valuable, yet it can be difficult to execute reliably in practice. Application constraints require systems to process data in real-time, not batches. Streaming data inherently exhibits concept drift, favoring algorithms that learn continuously. Furthermore, the massive number of independent streams in practice requires that anomaly detectors be fully automated. In this paper we propose a novel anomaly detection algorithm that meets these constraints. The technique is based on an online sequence memory algorithm called Hierarchical Temporal Memory (HTM). We also present results using the Numenta Anomaly Benchmark (NAB), a benchmark containing real-world data streams with labeled anomalies. The benchmark, the first of its kind, provides a controlled open-source environment for testing anomaly detection algorithms on streaming data. We present results and analysis for a wide range of algorithms on this benchmark, and discuss future challenges for the emerging field of streaming analytics.

📄 PDF Abstract BibTeX

Code (1)

numenta/NAB 공식 구현

Tasks

Anomaly DetectionTime SeriesTime Series Analysis

A Graph-based Framework for Online Time Series Anomaly Detection Using Model Ensemble

2026-01-04 · Zewei Yu, Jianqiu Xu, Caimin Li arxiv

With the increasing volume of streaming data in industrial systems, online anomaly detection has become a critical task. The diverse and rapidly evolving data patterns pose significant challenges for online anomaly detec…

Time Series Anomaly Detection Computational Efficiency Community Detection

Few-shot time-series anomaly detection with unsupervised domain adaptation

2023-11-01 · journal 2023 11 · Hongbo Li, Wenli Zheng, Feilong Tang, Yanmin Zhu 외

Anomaly detection for time-series data is crucial in the management of systems for streaming applications, computational services, and cloud platforms. The majority of current few-shot learning (FSL) approaches are suppo…

Anomaly Detection Domain Adaptation Few-Shot Learning Time Series+4

DeepAnT: A Deep Learning Approach for Unsupervised Anomaly Detection in Time Series

2018-12-19 · Mohsin Munir, Shoaib Ahmed Siddiqui, Andreas Dengel, Sheraz Ahmed

Traditional distance and density-based anomaly detection techniques are unable to detect periodic and seasonality related point anomalies which occur commonly in streaming data, leaving a big gap in time series anomaly…

Anomaly Detection Time Series Time Series Analysis Time Series Anomaly Detection+1

MemStream: Memory-Based Streaming Anomaly Detection

2021-06-07 · Siddharth Bhatia, Arjit Jain, Shivin Srivastava, Kenji Kawaguchi 외

Given a stream of entries over time in a multi-dimensional data setting where concept drift is present, how can we detect anomalous activities? Most of the existing unsupervised anomaly detection approaches seek to detec…

Anomaly Detection Denoising Unsupervised Anomaly Detection

Deep Autoencoders with Value-at-Risk Thresholding for Unsupervised Anomaly Detection

2019-12-09 · Albert Akhriev, Jakub Marecek

Many real-world monitoring and surveillance applications require non-trivial anomaly detection to be run in the streaming model. We consider an incremental-learning approach, wherein a deep-autoencoding (DAE) model of wh…

Anomaly Detection Incremental Learning subspace methods Unsupervised Anomaly Detection