NLP-ADBench: NLP Anomaly Detection Benchmark

2024-12-06 · Yuangang Li, Jiaqi Li, Zhuo Xiao, Tiankai Yang, Yi Nian, Xiyang Hu, Yue Zhao

Anomaly detection (AD) is a critical machine learning task with diverse applications in web systems, including fraud detection, content moderation, and user behavior analysis. Despite its significance, AD in natural language processing (NLP) remains underexplored, limiting advancements in detecting anomalies in text data such as harmful content, phishing attempts, or spam reviews. In this paper, we introduce NLP-ADBench, the most comprehensive benchmark for NLP anomaly detection (NLP-AD), comprising eight curated datasets and evaluations of nineteen state-of-the-art algorithms. These include three end-to-end methods and sixteen two-step algorithms that apply traditional anomaly detection techniques to language embeddings generated by bert-base-uncased and OpenAI's text-embedding-3-large models. Our results reveal critical insights and future directions for NLP-AD. Notably, no single model excels across all datasets, highlighting the need for automated model selection. Moreover, two-step methods leveraging transformer-based embeddings consistently outperform specialized end-to-end approaches, with OpenAI embeddings demonstrating superior performance over BERT embeddings. By releasing NLP-ADBench at https://github.com/USC-FORTIS/NLP-ADBench, we provide a standardized framework for evaluating NLP-AD methods, fostering the development of innovative approaches. This work fills a crucial gap in the field and establishes a foundation for advancing NLP anomaly detection, particularly in the context of improving the safety and reliability of web-based systems.

📄 PDF Abstract BibTeX arXiv:2412.04784

Code (1)

usc-fortis/nlp-adbench 공식 구현

Tasks

Anomaly Detection Fraud DetectionModel Selection

Methods 이 논문이 사용한 방법론

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Multi-Head Attention 설명 없음

Weight Decay 설명 없음

WordPiece 설명 없음

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

ADBench: Anomaly Detection Benchmark

2022-06-19 · Songqiao Han, Xiyang Hu, Hailiang Huang, Mingqi Jiang 외

Given a long list of anomaly detection algorithms developed in the last few decades, how do they perform with regard to (i) varying levels of supervision, (ii) different types of anomalies, and (iii) noisy and corrupted …

Anomaly Detection Outlier Detection

GADBench: Revisiting and Benchmarking Supervised Graph Anomaly Detection

2023-06-21 · NeurIPS 2023 11 · Jianheng Tang, Fengrui Hua, Ziqi Gao, Peilin Zhao 외

With a long history of traditional Graph Anomaly Detection (GAD) algorithms and recently popular Graph Neural Networks (GNNs), it is still not clear (1) how they perform under a standard comprehensive setting, (2) whethe…

Anomaly Detection Benchmarking Graph Anomaly Detection

FedTADBench: Federated Time-Series Anomaly Detection Benchmark

2022-12-19 · Fanxing Liu, Cheng Zeng, Le Zhang, Yingjie Zhou 외

Time series anomaly detection strives to uncover potential abnormal behaviors and patterns from temporal data, and has fundamental significance in diverse application scenarios. Constructing an effective detection model …

Anomaly Detection Federated Learning Time Series Time Series Analysis+1

Rethinking Weak Supervision in Anomaly Detection: A Comprehensive Benchmark

2026-05-25 · Xu Yao, Siyuan Zhou, Zhenbo Wu, Chaochuan Hou 외 arxiv

Weakly supervised anomaly detection (WSAD) has developed in three primary directions: incomplete, inexact, and inaccurate supervision. However, these directions remain isolated, lacking a unified framework to assess whet…

Supervised Anomaly Detection General Classification

Masked Diffusion Modeling for Anomaly Detection

2026-05-28 · Lixing Zhang, Yuchen Liang, Liyan Xie arxiv

Anomaly detection aims to identify samples that deviate from the nominal data distribution and is central to many safety-critical applications. However, developing effective anomaly detection methods for categorical, mix…

Anomaly Detection