Self-Supervised Losses for One-Class Textual Anomaly Detection
Current deep learning methods for anomaly detection in text rely on supervisory signals in inliers that may be unobtainable or bespoke architectures that are difficult to tune. We study a simpler alternative: fine-tuning Transformers on the inlier data with self-supervised objectives and using the losses as an anomaly score. Overall, the self-supervision approach outperforms other methods under various anomaly detection scenarios, improving the AUROC score on semantic anomalies by 11.6% and on syntactic anomalies by 22.8% on average. Additionally, the optimal objective and resultant learnt representation depend on the type of downstream anomaly. The separability of anomalies and inliers signals that a representation is more effective for detecting semantic anomalies, whilst the presence of narrow feature directions signals a representation that is effective for detecting syntactic anomalies.
Code (0)
등록된 구현이 없습니다.
Tasks
Anomaly DetectionSimilar Papers 제목 키워드 기반
Self-Supervised Likelihood Estimation with Energy Guidance for Anomaly Segmentation in Urban Scenes
Robust autonomous driving requires agents to accurately identify unexpected areas (anomalies) in urban scenes. To this end, some critical issues remain open: how to design advisable metric to measure anomalies, and how t…
Anomaly DetectionAnomaly SegmentationAutonomous DrivingAnoSeg: Anomaly Segmentation Network Using Self-Supervised Learning
Anomaly segmentation, which localizes defective areas, is an important component in large-scale industrial manufacturing. However, most recent researches have focused on anomaly detection. This paper proposes a novel ano…
Anomaly DetectionAnomaly SegmentationSegmentationSelf-Supervised LearningDSV: An Alignment Validation Loss for Self-supervised Outlier Model Selection
Self-supervised learning (SSL) has proven effective in solving various problems by generating internal supervisory signals. Unsupervised anomaly detection, which faces the high cost of obtaining true labels, is an area t…
Anomaly DetectionData AugmentationModel SelectionSelf-Supervised Learning+1Self-Attentive, Multi-Context One-Class Classification for Unsupervised Anomaly Detection on Text
There exist few text-specific methods for unsupervised anomaly detection, and for those that do exist, none utilize pre-trained models for distributed vector representations of words. In this paper we introduce a new ano…
Anomaly DetectionContextual Anomaly DetectionGeneral ClassificationOne-Class Classification+2Multi-Task Self-Supervised Time-Series Representation Learning
Time-series representation learning can extract representations from data with temporal dynamics and sparse labels. When labeled data are sparse but unlabeled data are abundant, contrastive learning, i.e., a framework to…
Anomaly DetectionContrastive LearningMulti-Task LearningRepresentation Learning+4