AI Safety in Practice: Enhancing Adversarial Robustness in Multimodal Image Captioning
Multimodal machine learning models that combine visual and textual data are increasingly being deployed in critical applications, raising significant safety and security concerns due to their vulnerability to adversarial attacks. This paper presents an effective strategy to enhance the robustness of multimodal image captioning models against such attacks. By leveraging the Fast Gradient Sign Method (FGSM) to generate adversarial examples and incorporating adversarial training techniques, we demonstrate improved model robustness on two benchmark datasets: Flickr8k and COCO. Our findings indicate that selectively training only the text decoder of the multimodal architecture shows performance comparable to full adversarial training while offering increased computational efficiency. This targeted approach suggests a balance between robustness and training costs, facilitating the ethical deployment of multimodal AI systems across various domains.
Code (0)
등록된 구현이 없습니다.
Tasks
Adversarial RobustnessComputational EfficiencyDecoderImage CaptioningSimilar Papers 제목 키워드 기반
Multimodal Large Language Models for Enhanced Traffic Safety: A Comprehensive Review and Future Trends
Traffic safety remains a critical global challenge, with traditional Advanced Driver-Assistance Systems (ADAS) often struggling in dynamic real-world scenarios due to fragmented sensor processing and susceptibility to ad…
Adversarial RobustnessDecision MakingScene UnderstandingMMT-ARD: Multimodal Multi-Teacher Adversarial Distillation for Robust Vision-Language Models
Vision-Language Models (VLMs) are increasingly deployed in safety-critical applications, making their adversarial robustness a crucial concern. While adversarial knowledge distillation has shown promise in transferring r…
Adversarial RobustnessKnowledge DistillationYuvion VL: A Multimodal Foundation Model for Adversarial Content and AI Safety
General-purpose models often struggle to reliably identify and understand real-world multimodal risks, largely due to the inherent multimodal adversarial nature of content and AI safety. We present Yuvion VL, a family of…
Adversarial RobustnessCalibrated Adversarial Sampling: Multi-Armed Bandit-Guided Generalization Against Unforeseen Attacks
Deep Neural Networks (DNNs) are known to be vulnerable to various adversarial perturbations. To address the safety concerns arising from these vulnerabilities, adversarial training (AT) has emerged as one of the most eff…
Revisiting the Adversarial Robustness of Vision Language Models: a Multimodal Perspective
Pretrained vision-language models (VLMs) like CLIP exhibit exceptional generalization across diverse downstream tasks. While recent studies reveal their vulnerability to adversarial attacks, research to date has primaril…
Adversarial DefenseAdversarial RobustnessAdversarial Text