VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model

2024-06-20 · Sibo Wang, Xiangkui Cao, Jie Zhang, Zheng Yuan, Shiguang Shan, Xilin Chen, Wen Gao

The emergence of Large Vision-Language Models (LVLMs) marks significant strides towards achieving general artificial intelligence. However, these advancements are accompanied by concerns about biased outputs, a challenge that has yet to be thoroughly explored. Existing benchmarks are not sufficiently comprehensive in evaluating biases due to their limited data scale, single questioning format and narrow sources of bias. To address this problem, we introduce VLBiasBench, a comprehensive benchmark designed to evaluate biases in LVLMs. VLBiasBench, features a dataset that covers nine distinct categories of social biases, including age, disability status, gender, nationality, physical appearance, race, religion, profession, social economic status, as well as two intersectional bias categories: race x gender and race x social economic status. To build a large-scale dataset, we use Stable Diffusion XL model to generate 46,848 high-quality images, which are combined with various questions to creat 128,342 samples. These questions are divided into open-ended and close-ended types, ensuring thorough consideration of bias sources and a comprehensive evaluation of LVLM biases from multiple perspectives. We conduct extensive evaluations on 15 open-source models as well as two advanced closed-source models, yielding new insights into the biases present in these models. Our benchmark is available at https://github.com/Xiangkui-Cao/VLBiasBench.

📄 PDF Abstract BibTeX arXiv:2406.14194

Code (1)

xiangkui-cao/vlbiasbench 공식 구현 pytorch

Tasks

Language ModelingLanguage Modelling

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

FairCoder: Evaluating Social Bias of LLMs in Code Generation

2025-01-09 · Yongkang Du, Jen-tse Huang, Jieyu Zhao, Lu Lin

Large language models (LLMs) have been widely deployed in coding tasks, drawing increasing attention to the evaluation of the quality and safety of LLMs' outputs. However, research on bias in code generation remains limi…

Code Generation Fairness

JudgeLM: Fine-tuned Large Language Models are Scalable Judges

2023-10-26 · Lianghui Zhu, Xinggang Wang, Xinlong Wang

Evaluating Large Language Models (LLMs) in open-ended scenarios is challenging because existing benchmarks and metrics can not measure them comprehensively. To address this problem, we propose to fine-tune LLMs as scalab…

Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey

2022-07-14 · Max Hort, Zhenpeng Chen, Jie M. Zhang, Mark Harman 외

This paper provides a comprehensive survey of bias mitigation methods for achieving fairness in Machine Learning (ML) models. We collect a total of 341 publications concerning bias mitigation for ML classifiers. These me…

Benchmarking BIG-bench Machine Learning Fairness Survey

From Lists to Emojis: How Format Bias Affects Model Alignment

2024-09-18 · Xuanchang Zhang, Wei Xiong, Lichang Chen, Tianyi Zhou 외

In this paper, we study format biases in reinforcement learning from human feedback (RLHF). We observe that many widely-used preference models, including human evaluators, GPT-4, and top-ranking models on the RewardBench…

Chatbot

Evaluating LLMs for Demographic-Targeted Social Bias Detection: A Comprehensive Benchmark Study

2025-10-06 · Ayan Majumdar, Feihao Chen, Jinghui Li, Xiaozhen Wang arxiv

Large-scale web-scraped text corpora used to train general-purpose AI models often contain harmful demographic-targeted social biases, creating a regulatory need for data auditing and developing scalable bias-detection m…

Bias Detection