GradStop: Exploring Training Dynamics in Unsupervised Outlier Detection through Gradient
Unsupervised Outlier Detection (UOD) is a critical task in data mining and machine learning, aiming to identify instances that significantly deviate from the majority. Without any label, deep UOD methods struggle with the misalignment between the model's direct optimization goal and the final performance goal of Outlier Detection (OD) task. Through the perspective of training dynamics, this paper proposes an early stopping algorithm to optimize the training of deep UOD models, ensuring they perform optimally in OD rather than overfitting the entire contaminated dataset. Inspired by UOD mechanism and inlier priority phenomenon, where intuitively models fit inliers more quickly than outliers, we propose GradStop, a sampling-based label-free algorithm to estimate model's real-time performance during training. First, a sampling method generates two sets: one likely containing more outliers and the other more inliers, then a metric based on gradient cohesion is applied to probe into current training dynamics, which reflects model's performance on OD task. Experimental results on 4 deep UOD algorithms and 47 real-world datasets and theoretical proofs demonstrate the effectiveness of our proposed early stopping algorithm in enhancing the performance of deep UOD models. Auto Encoder (AE) enhanced by GradStop achieves better performance than itself, other SOTA UOD methods, and even ensemble AEs. Our method provides a robust and effective solution to the problem of performance degradation during training, enabling deep UOD models to achieve better potential in anomaly detection tasks.
Code (1)
Tasks
Anomaly DetectionOutlier DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
GRADSTOP: Early Stopping of Gradient Descent via Posterior Sampling
Machine learning models are often learned by minimising a loss function on the training data using a gradient descent algorithm. These models often suffer from overfitting, leading to a decline in predictive performance …
Transfer LearningPost-Hoc Calibrated Anomaly Detection
Deep unsupervised anomaly detection has seen improvements in a supervised binary classification paradigm in which auxiliary external data is included in the training set as anomalous data in a process referred to as outl…
Anomaly DetectionBinary ClassificationUnsupervised Anomaly DetectionExploring the Limits of Out-of-Distribution Detection
Near out-of-distribution detection (OOD) is a major challenge for deep neural networks. We demonstrate that large-scale pre-trained transformers can significantly improve the state-of-the-art (SOTA) on a range of near OO…
Out-of-Distribution DetectionOut of Distribution (OOD) DetectionUnsupervised Pre-trainingUnsupervised Deep One-Class Classification with Adaptive Threshold based on Training Dynamics
One-class classification has been a prevailing method in building deep anomaly detection models under the assumption that a dataset consisting of normal samples is available. In practice, however, abnormal samples are of…
Anomaly DetectionOne-Class ClassificationOutlier DetectionUnsupervised Embedding Learning from Uncertainty Momentum Modeling
Existing popular unsupervised embedding learning methods focus on enhancing the instance-level local discrimination of the given unlabeled images by exploring various negative data. However, the existed sample outliers w…