Do VLMs Have a Moral Backbone? A Study on the Fragile Morality of Vision-Language Models
Despite substantial efforts toward improving the moral alignment of Vision-Language Models (VLMs), it remains unclear whether their ethical judgments are stable in realistic settings. This work studies moral robustness in VLMs, defined as the ability to preserve moral judgments under textual and visual perturbations that do not alter the underlying moral context. We systematically probe VLMs with a diverse set of model-agnostic multimodal perturbations and find that their moral stances are highly fragile, frequently flipping under simple manipulations. Our analysis reveals systematic vulnerabilities across perturbation types, moral domains, and model scales, including a sycophancy trade-off where stronger instruction-following models are more susceptible to persuasion. We further show that lightweight inference-time interventions can partially restore moral stability. These results demonstrate that moral alignment alone is insufficient and that moral robustness is a necessary criterion for the responsible deployment of VLMs.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Moral Sycophancy in Vision Language Models
Sycophancy in Vision-Language Models (VLMs) refers to their tendency to align with user opinions, often at the expense of moral or factual accuracy. While prior studies have explored sycophantic behavior in general conte…
M$^3$oralBench: A MultiModal Moral Benchmark for LVLMs
Recently, large foundation models, including large language models (LLMs) and large vision-language models (LVLMs), have become essential tools in critical fields such as law, finance, and healthcare. As these models inc…
Moral ScenariosMM-SCALE: Grounded Multimodal Moral Reasoning via Scalar Judgment and Listwise Alignment
Vision-Language Models (VLMs) continue to struggle to make morally salient judgments in multimodal and socially ambiguous contexts. Prior works typically rely on binary or pairwise supervision, which often fail to captur…
ProMoral-Bench: Evaluating Prompting Strategies for Moral Reasoning and Safety in LLMs
Prompt design significantly impacts the moral competence and safety alignment of large language models (LLMs), yet empirical comparisons remain fragmented across datasets and models.We introduce ProMoral-Bench, a unified…
Prompt EngineeringMORALISE: A Structured Benchmark for Moral Alignment in Visual Language Models
Warning: This paper contains examples of harmful language and images. Reader discretion is advised. Recently, vision-language models have demonstrated increasing influence in morally sensitive domains such as autonomous …
Autonomous DrivingMultimodal Reasoning