Quantifying Societal Bias Amplification in Image Captioning
We study societal bias amplification in image captioning. Image captioning models have been shown to perpetuate gender and racial biases, however, metrics to measure, quantify, and evaluate the societal bias in captions are not yet standardized. We provide a comprehensive study on the strengths and limitations of each metric, and propose LIC, a metric to study captioning bias amplification. We argue that, for image captioning, it is not enough to focus on the correct prediction of the protected attribute, and the whole context should be taken into account. We conduct extensive evaluation on traditional and state-of-the-art image captioning models, and surprisingly find that, by only focusing on the protected attribute prediction, bias mitigation models are unexpectedly amplifying bias.
Code (1)
Tasks
AttributeImage CaptioningSimilar Papers 제목 키워드 기반
Reproducibility Study of “Quantifying Societal Bias Amplification in Image Captioning”
Scope of reproducibility - We study the reproducibility of the paper "Quantifying Societal Bias Amplification in Image Captioning" by Hirota et al. In this paper, the authors propose a new metric to measure bias amplific…
RELIC: Reproducibility and Extension on LIC metric in quantifying bias in captioning models
Scope of Reproducibility In this work we reproduce and extend the results presented in “Quantifying Societal Bias Amplification in Image Captioning” by Hirota et al. This paper introduces LIC, a metric to quantify bias a…
ImageCaptioner$^2$: Image Captioner for Image Captioning Bias Amplification Assessment
Most pre-trained learning systems are known to suffer from bias, which typically emerges from the data, the model, or both. Measuring and quantifying bias and its sources is a challenging task and has been extensively st…
Image CaptioningMeasuring directional bias amplification in image captions using predictability
When we train models on biased ML datasets, they not only learn these biases but can inflate them at test time - a phenomenon called bias amplification. To measure bias amplification in ML datasets, many co-occurrence-ba…
Image Captioningimage-classificationImage ClassificationResampled Datasets Are Not Enough: Mitigating Societal Bias Beyond Single Attributes
We tackle societal bias in image-text datasets by removing spurious correlations between protected groups and image attributes. Traditional methods only target labeled attributes, ignoring biases from unlabeled ones. Usi…
Image Captioningimage-classificationImage ClassificationMulti-Label Image Classification