paper-with-me

홈 › Papers

Contrastive Language-Vision AI Models Pretrained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Bias

2022-12-21 · Robert Wolfe, Yiwei Yang, Bill Howe, Aylin Caliskan

Nine language-vision AI models trained on web scrapes with the Contrastive Language-Image Pretraining (CLIP) objective are evaluated for evidence of a bias studied by psychologists: the sexual objectification of girls and women, which occurs when a person's human characteristics, such as emotions, are disregarded and the person is treated as a body. We replicate three experiments in psychology quantifying sexual objectification and show that the phenomena persist in AI. A first experiment uses standardized images of women from the Sexual OBjectification and EMotion Database, and finds that human characteristics are disassociated from images of objectified women: the model's recognition of emotional state is mediated by whether the subject is fully or partially clothed. Embedding association tests (EATs) return significant effect sizes for both anger (d >0.80) and sadness (d >0.50), associating images of fully clothed subjects with emotions. GRAD-CAM saliency maps highlight that CLIP gets distracted from emotional expressions in objectified images. A second experiment measures the effect in a representative application: an automatic image captioner (Antarctic Captions) includes words denoting emotion less than 50% as often for images of partially clothed women than for images of fully clothed women. A third experiment finds that images of female professionals (scientists, doctors, executives) are likely to be associated with sexual descriptions relative to images of male professionals. A fourth experiment shows that a prompt of "a [age] year old girl" generates sexualized images (as determined by an NSFW classifier) up to 73% of the time for VQGAN-CLIP and Stable Diffusion; the corresponding rate for boys never surpasses 9%. The evidence indicates that language-vision AI models trained on web scrapes learn biases of sexual objectification, which propagate to downstream applications.

📄 PDF Abstract BibTeX arXiv:2212.11261

Code (1)

yanyiwei/clip-gender-bias 공식 구현

Methods 이 논문이 사용한 방법론

CLIP Contrastive Language-Image Pre-training (CLIP), consisting of a simplified version of ConVIRT trained from scratch, is an efficient method of image representation learning…
Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Similar Papers 제목 키워드 기반

LEMoN: Label Error Detection using Multimodal Neighbors

2024-07-10 · Haoran Zhang, Aparna Balagopalan, Nassim Oufattole, Hyewon Jeong 외

Large repositories of image-caption pairs are essential for the development of vision-language models. However, these datasets are often extracted from noisy data scraped from the web, and contain many mislabeled instanc…

Label Error Detection

X-Tokenizer: A Multimodal Action Tokenizer for Vision-Language-Action Pretraining

2026-06-07 · Miracle Kang, Lights Shi, Lucy Liang, Roy Gan 외 arxiv

Modern Vision-Language-Action (VLA) models must bridge pretrained vision-language reasoning and precise continuous robot control. Existing action tokenizers discretize actions primarily for reconstruction, producing code…

Multimodal Reasoning

CALM: Contrastive Aligned Audio-Language Multirate and Multimodal Representations

2022-02-08 · Vin Sachidananda, Shao-Yen Tseng, Erik Marchi, Sachin Kajarekar 외

Deriving multimodal representations of audio and lexical inputs is a central problem in Natural Language Understanding (NLU). In this paper, we present Contrastive Aligned Audio-Language Multirate and Multimodal Represen…

Emotion RecognitionNatural Language Understanding

FLAVARS: A Multimodal Foundational Language and Vision Alignment Model for Remote Sensing

2025-01-14 · Isaac Corley, Simone Fobi Nsutezo, Anthony Ortiz, Caleb Robinson 외

Remote sensing imagery is dense with objects and contextual visual information. There is a recent trend to combine paired satellite images and text captions for pretraining performant encoders for downstream tasks. Howev…

ClassificationContrastive LearningSemantic Segmentationzero-shot-classification+1

PaLI-3 Vision Language Models: Smaller, Faster, Stronger

2023-10-13 · Xi Chen, Xiao Wang, Lucas Beyer, Alexander Kolesnikov 외

This paper presents PaLI-3, a smaller, faster, and stronger vision language model (VLM) that compares favorably to similar models that are 10x larger. As part of arriving at this strong performance, we compare Vision Tra…

Chart Question AnsweringCross-Modal Retrievalimage-classificationImage Classification+6