Iterative Adversarial Attack on Image-guided Story Ending Generation
Multimodal learning involves developing models that can integrate information from various sources like images and texts. In this field, multimodal text generation is a crucial aspect that involves processing data from multiple modalities and outputting text. The image-guided story ending generation (IgSEG) is a particularly significant task, targeting on an understanding of complex relationships between text and image data with a complete story text ending. Unfortunately, deep neural networks, which are the backbone of recent IgSEG models, are vulnerable to adversarial samples. Current adversarial attack methods mainly focus on single-modality data and do not analyze adversarial attacks for multimodal text generation tasks that use cross-modal information. To this end, we propose an iterative adversarial attack method (Iterative-attack) that fuses image and text modality attacks, allowing for an attack search for adversarial text and image in an more effective iterative way. Experimental results demonstrate that the proposed method outperforms existing single-modal and non-iterative multimodal attack methods, indicating the potential for improving the adversarial robustness of multimodal text generation models, such as multimodal machine translation, multimodal question answering, etc.
Code (0)
등록된 구현이 없습니다.
Tasks
Adversarial AttackAdversarial RobustnessAdversarial TextImage-guided Story Ending GenerationMachine TranslationMultimodal Machine TranslationQuestion AnsweringText GenerationSimilar Papers 제목 키워드 기반
Semi-supervised Semantics-guided Adversarial Training for Trajectory Prediction
Predicting the trajectories of surrounding objects is a critical task for self-driving vehicles and many other autonomous systems. Recent works demonstrate that adversarial attacks on trajectory prediction, where small c…
Adversarial RobustnessDecision MakingPredictionTrajectory PredictionSemi-supervised Semantics-guided Adversarial Training for Robust Trajectory Prediction
Predicting the trajectories of surrounding objects is a critical task for self-driving vehicles and many other autonomous systems. Recent works demonstrate that adversarial attacks on trajectory prediction, where sma…
Decision MakingPredictionTrajectory PredictionBenchmarking Factual Robustness of LLMs via Multi-conversation Persuasion
As Large Language Models (LLMs) increasingly serve as primary knowledge retrieval interfaces, their robustness against \textit{persuasion attacks}---attempts to inject misinformation or enforce counterfactuals---has beco…
Aha! Adaptive History-Driven Attack for Decision-Based Black-Box Models
The decision-based black-box attack means to craft adversarial examples with only the top-1 label of the victim model available. A common practice is to start from a large perturbation and then iteratively reduce it …
Dimensionality ReductionDefense-guided Transferable Adversarial Attacks
Though deep neural networks perform challenging tasks excellently, they are susceptible to adversarial examples, which mislead classifiers by applying human-imperceptible perturbations on clean inputs. Under the query-fr…
Adversarial Attack