Reducing Events to Augment Log-based Anomaly Detection Models: An Empirical Study
As software systems grow increasingly intricate, the precise detection of anomalies have become both essential and challenging. Current log-based anomaly detection methods depend heavily on vast amounts of log data leading to inefficient inference and potential misguidance by noise logs. However, the quantitative effects of log reduction on the effectiveness of anomaly detection remain unexplored. Therefore, we first conduct a comprehensive study on six distinct models spanning three datasets. Through the study, the impact of log quantity and their effectiveness in representing anomalies is qualifies, uncovering three distinctive log event types that differently influence model performance. Drawing from these insights, we propose LogCleaner: an efficient methodology for the automatic reduction of log events in the context of anomaly detection. Serving as middleware between software systems and models, LogCleaner continuously updates and filters anti-events and duplicative-events in the raw generated logs. Experimental outcomes highlight LogCleaner's capability to reduce over 70% of log events in anomaly detection, accelerating the model's inference speed by approximately 300%, and universally improving the performance of models for anomaly detection.
Code (0)
등록된 구현이 없습니다.
Tasks
Anomaly DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Boosting Anomaly Detection Using Unsupervised Diverse Test-Time Augmentation
Anomaly detection is a well-known task that involves the identification of abnormal events that occur relatively infrequently. Methods for improving anomaly detection performance have been widely studied. However, no stu…
Anomaly DetectionUBnormal: New Benchmark for Supervised Open-Set Video Anomaly Detection
Detecting abnormal events in video is commonly framed as a one-class classification task, where training videos contain only normal events, while test videos encompass both normal and abnormal events. In this scenario, a…
Action RecognitionAnomaly DetectionEvent DetectionOne-Class Classification+1KRONE: Scalable LLM-Augmented Log Anomaly Detection via Hierarchical Abstraction
Log anomaly detection is crucial for uncovering system failures and security risks. Although logs originate from nested component executions with clear boundaries, this structure is lost when stored as flat sequences. As…
Anomaly DetectionRAD: Rule-Augmented Relational Anomaly Detection
Anomaly detection is often applied to data stored in relational databases, yet most existing methods require flattening multiple tables into a single feature matrix. This flattening can obscure entity identity, schema st…
Graph Representation LearningAnomaly DetectionA2Log: Attentive Augmented Log Anomaly Detection
Anomaly detection becomes increasingly important for the dependability and serviceability of IT services. As log lines record events during the execution of IT services, they are a primary source for diagnostics. Thereby…
Anomaly DetectionData AugmentationUnsupervised Anomaly Detection