Sequential Multivariate Change Detection with Calibrated and Memoryless False Detection Rates
Responding appropriately to the detections of a sequential change detector requires knowledge of the rate at which false positives occur in the absence of change. Setting detection thresholds to achieve a desired false positive rate is challenging. Existing works resort to setting time-invariant thresholds that focus on the expected runtime of the detector in the absence of change, either bounding it loosely from below or targeting it directly but with asymptotic arguments that we show cause significant miscalibration in practice. We present a simulation-based approach to setting time-varying thresholds that allows a desired expected runtime to be accurately targeted whilst additionally keeping the false positive rate constant across time steps. Whilst the approach to threshold setting is metric agnostic, we show how the cost of using the popular quadratic time MMD estimator can be reduced from $O(N^2B)$ to $O(N^2+NB)$ during configuration and from $O(N^2)$ to $O(N)$ during operation, where $N$ and $B$ are the numbers of reference and bootstrap samples respectively.
Code (1)
Tasks
Change DetectionSimilar Papers 제목 키워드 기반
Partially-Observable Sequential Change-Point Detection for Autocorrelated Data via Upper Confidence Region
Sequential change point detection for multivariate autocorrelated data is a very common problem in practice. However, when the sensing resources are limited, only a subset of variables from the multivariate system can be…
Change Point DetectionMultivariate Prediction Intervals for Random Forests
Accurate uncertainty estimates can significantly improve the performance of iterative design of experiments, as in Sequential and Reinforcement learning. For many such problems in engineering and the physical sciences, t…
PredictionPrediction IntervalsReinforcement Learning (RL)Change Point Detection via Multivariate Singular Spectrum Analysis
The objective of change point detection (CPD) is to detect significant and abrupt changes in the dynamics of the underlying system of interest through multivariate time series observations. In this work, we develop and a…
Change Point DetectionTime SeriesTime Series AnalysisABACUS: Unsupervised Multivariate Change Detection via Bayesian Source Separation
Change detection involves segmenting sequential data such that observations in the same segment share some desired properties. Multivariate change detection continues to be a challenging problem due to the variety of way…
Change DetectionDimensionality ReductionGradient-Free Warm-Start Library Recovery: an Amortized-Regret Separation
Continual learning that is gradient-free, local, online, and append-only is attractive for edge and streaming deployment, but its value is usually argued informally. We give a provable account on recurring-regime streams…
Continual Learning