Backdoor Learning on Sequence to Sequence Models

2023-05-03 · Lichang Chen, Minhao Cheng, Heng Huang

Backdoor learning has become an emerging research area towards building a trustworthy machine learning system. While a lot of works have studied the hidden danger of backdoor attacks in image or text classification, there is a limited understanding of the model's robustness on backdoor attacks when the output space is infinite and discrete. In this paper, we study a much more challenging problem of testing whether sequence-to-sequence (seq2seq) models are vulnerable to backdoor attacks. Specifically, we find by only injecting 0.2\% samples of the dataset, we can cause the seq2seq model to generate the designated keyword and even the whole sentence. Furthermore, we utilize Byte Pair Encoding (BPE) to create multiple new triggers, which brings new challenges to backdoor detection since these backdoors are not static. Extensive experiments on machine translation and text summarization have been conducted to show our proposed methods could achieve over 90\% attack success rate on multiple datasets and models.

📄 PDF Abstract BibTeX arXiv:2305.02424

Code (0)

등록된 구현이 없습니다.

Tasks

Machine TranslationSentencetext-classification Text Classification Text Summarization

Methods 이 논문이 사용한 방법론

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

Seq2Seq Seq2Seq, or Sequence To Sequence, is a model used in sequence prediction tasks, such as language modelling and machine translation. The idea is to use one…

BadHMP: Backdoor Attack against Human Motion Prediction

2024-09-29 · Chaohui Xu, Si Wang, Chip-Hong Chang

Precise future human motion prediction over subsecond horizons from past observations is crucial for various safety-critical applications. To date, only one study has examined the vulnerability of human motion prediction…

Backdoor Attack Human motion prediction motion prediction Prediction

ConfGuard: A Simple and Effective Backdoor Detection for Large Language Models

2025-08-02 · Zihan Wang, Rui Zhang, Hongwei Li, Wenshu Fan 외 arxiv

Backdoor attacks pose a significant threat to Large Language Models (LLMs), where adversaries can embed hidden triggers to manipulate LLM's outputs. Most existing defense methods, primarily designed for classification ta…

Spinning Sequence-to-Sequence Models with Meta-Backdoors

2021-07-22 · Eugene Bagdasaryan, Vitaly Shmatikov

We investigate a new threat to neural sequence-to-sequence (seq2seq) models: training-time attacks that cause models to "spin" their output and support a certain sentiment when the input contains adversary-chosen trigger…

Sentiment Analysis

Defending against Backdoor Attacks in Natural Language Generation

2022-01-16 · ACL ARR January 2022 1 · Anonymous

The frustratingly fragile nature of neural network models make current natural language generation (NLG) systems prone to backdoor attacks and generate malicious sequences that could be sexist or offensive. Unfortunatel…

Backdoor Attack Dialogue Generation Machine Translation Text Generation+1

BackdoorIDS: Zero-shot Backdoor Detection for Pretrained Vision Encoder

2026-03-12 · Siquan Huang, Yijiang Li, Ningzhi Gao, Xingfu Yan 외 arxiv

Self-supervised and multimodal vision encoders learn strong visual representations that are widely adopted in downstream vision tasks and large vision-language models (LVLMs). However, downstream users often rely on thir…