Solving Trojan Detection Competitions with Linear Weight Classification

2024-11-05 · Todd Huster, Peter Lin, Razvan Stefanescu, Emmanuel Ekwedike, Ritu Chadha

Neural networks can conceal malicious Trojan backdoors that allow a trigger to covertly change the model behavior. Detecting signs of these backdoors, particularly without access to any triggered data, is the subject of ongoing research and open challenges. In one common formulation of the problem, we are given a set of clean and poisoned models and need to predict whether a given test model is clean or poisoned. In this paper, we introduce a detector that works remarkably well across many of the existing datasets and domains. It is obtained by training a binary classifier on a large number of models' weights after performing a few different pre-processing steps including feature selection and standardization, reference model weights subtraction, and model alignment prior to detection. We evaluate this algorithm on a diverse set of Trojan detection benchmarks and domains and examine the cases where the approach is most and least effective.

📄 PDF Abstract BibTeX arXiv:2411.03445

Code (0)

등록된 구현이 없습니다.

Tasks

Classificationfeature selection

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Feature Selection Feature selection, also known as variable selection, attribute selection or variable subset selection, is the process of selecting a subset of relevant features (variables,…

Trojan Signatures in DNN Weights

2021-09-07 · Greg Fields, Mohammad Samragh, Mojan Javaheripi, Farinaz Koushanfar 외

Deep neural networks have been shown to be vulnerable to backdoor, or trojan, attacks where an adversary has embedded a trigger in the network at training time such that the model correctly classifies all standard inputs…

Trojan horse hunt in deep forecasting models: Insights from the European Space Agency competition

2026-03-20 · Krzysztof Kotowski, Ramez Shendy, Jakub Nalepa, Agata Kaczmarek 외 arxiv

Forecasting plays a crucial role in modern safety-critical applications, such as space operations. However, the increasing use of deep forecasting models introduces a new security risk of trojan horse attacks, carried ou…

Time Series Forecasting

On Trojan Signatures in Large Language Models of Code

2024-02-23 · Aftab Hussain, Md Rafiqul Islam Rabin, Mohammad Amin Alipour

Trojan signatures, as described by Fields et al. (2021), are noticeable differences in the distribution of the trojaned class parameters (weights) and the non-trojaned class parameters of the trojaned model, that can be …

Binary Classification Defect Detection

TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems

2019-08-02 · Wenbo Guo, Lun Wang, Xinyu Xing, Min Du 외

A trojan backdoor is a hidden pattern typically implanted in a deep neural network. It could be activated and thus forces that infected model behaving abnormally only when an input data sample with a particular trigger p…

Anomaly Detection

Dormant Neural Trojans

2022-11-02 · Feisi Fu, Panagiota Kiourti, Wenchao Li

We present a novel methodology for neural network backdoor attacks. Unlike existing training-time attacks where the Trojaned network would respond to the Trojan trigger after training, our approach inserts a Trojan that …