AVScan2Vec: Feature Learning on Antivirus Scan Data for Production-Scale Malware Corpora
When investigating a malicious file, searching for related files is a common task that malware analysts must perform. Given that production malware corpora may contain over a billion files and consume petabytes of storage, many feature extraction and similarity search approaches are computationally infeasible. Our work explores the potential of antivirus (AV) scan data as a scalable source of features for malware. This is possible because AV scan reports are widely available through services such as VirusTotal and are ~100x smaller than the average malware sample. The information within an AV scan report is abundant with information and can indicate a malicious file's family, behavior, target operating system, and many other characteristics. We introduce AVScan2Vec, a language model trained to comprehend the semantics of AV scan data. AVScan2Vec ingests AV scan data for a malicious file and outputs a meaningful vector representation. AVScan2Vec vectors are ~3 to 85x smaller than popular alternatives in use today, enabling faster vector comparisons and lower memory usage. By incorporating Dynamic Continuous Indexing, we show that nearest-neighbor queries on AVScan2Vec vectors can scale to even the largest malware production datasets. We also demonstrate that AVScan2Vec vectors are superior to other leading malware feature vector representations across nearly all classification, clustering, and nearest-neighbor lookup algorithms that we evaluated.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModellingSimilar Papers 제목 키워드 기반
Adapting Novelty towards Generating Antigens for Antivirus systems
It is well known that anti-malware scanners depend on malware signatures to identify malware. However, even minor modifications to malware code structure results in a change in the malware signature thus enabling the var…
Evolutionary AlgorithmsMalware AnalysisMalware DetectionRank-1 Similarity Matrix Decomposition For Modeling Changes in Antivirus Consensus Through Time
Although groups of strongly correlated antivirus engines are known to exist, at present there is limited understanding of how or why these correlations came to be. Using a corpus of 25 million VirusTotal reports represen…
On Improving Antivirus Scanning Engines: Memory On- Access Scanner
The Antivirus (AV) products are utilized by home user's community to attain protection. To some extent, the AV meets users' expectations by detecting previously known malware samples. In this study, we question the set o…
A Malware Obfuscation AI Technique to Evade Antivirus Detection in Counter Forensic Domain
Data obfuscation is the process of converting a constant or a variable into computational results of several constants to make it hard for the files to be detected or analyzed by anti-malware engines. In recent years, ma…
Benchmark Static API Call Datasets for Malware Family Classification
Nowadays, malware and malware incidents are increasing daily, even with various antivirus systems and malware detection or classification methodologies. Machine learning techniques have been the main focus of the securit…
ClassificationMalware ClassificationMalware Detection