paper-with-me

홈 › Papers

On Background Bias of Post-Hoc Concept Embeddings in Computer Vision DNNs

2025-04-11 · Gesina Schwalbe, Georgii Mikriukov, Edgar Heinert, Stavros Gerolymatos, Mert Keser, Alois Knoll, Matthias Rottmann, Annika Mütze

The thriving research field of concept-based explainable artificial intelligence (C-XAI) investigates how human-interpretable semantic concepts embed in the latent spaces of deep neural networks (DNNs). Post-hoc approaches therein use a set of examples to specify a concept, and determine its embeddings in DNN latent space using data driven techniques. This proved useful to uncover biases between different target (foreground or concept) classes. However, given that the background is mostly uncontrolled during training, an important question has been left unattended so far: Are/to what extent are state-of-the-art, data-driven post-hoc C-XAI approaches themselves prone to biases with respect to their backgrounds? E.g., wild animals mostly occur against vegetation backgrounds, and they seldom appear on roads. Even simple and robust C-XAI methods might abuse this shortcut for enhanced performance. A dangerous performance degradation of the concept-corner cases of animals on the road could thus remain undiscovered. This work validates and thoroughly confirms that established Net2Vec-based concept segmentation techniques frequently capture background biases, including alarming ones, such as underperformance on road scenes. For the analysis, we compare 3 established techniques from the domain of background randomization on >50 concepts from 2 datasets, and 7 diverse DNN architectures. Our results indicate that even low-cost setups can provide both valuable insight and improved background robustness.

📄 PDF Abstract BibTeX arXiv:2504.08602

Code (1)

gesina/bg_randomized_loce 공식 구현 pytorch

Tasks

Explainable artificial intelligence

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Similar Papers 제목 키워드 기반

Conceptor Debiasing of Word Representations Evaluated on WEAT

2019-06-14 · WS 2019 8 · Saket Karve, Lyle Ungar, João Sedoc

Bias in word embeddings such as Word2Vec has been widely investigated, and many efforts made to remove such bias. We show how to use conceptors debiasing to post-process both traditional and contextualized word embedding…

Word Embeddings

Conceptor-Aided Debiasing of Large Language Models

2022-11-20 · Li S. Yifei, Lyle Ungar, João Sedoc

Pre-trained large language models (LLMs) reflect the inherent social biases of their training corpus. Many methods have been proposed to mitigate this issue, but they often fail to debias or they sacrifice model accuracy…

Language Modelling

Robustness and Reliability of Gender Bias Assessment in Word Embeddings: The Role of Base Pairs

2020-10-06 · Asian Chapter of the Association for Computational Linguistics 2020 · Haiyang Zhang, Alison Sneyd, Mark Stevenson

It has been shown that word embeddings can exhibit gender bias, and various methods have been proposed to quantify this. However, the extent to which the methods are capturing social stereotypes inherited from the data h…

Word EmbeddingsWord Similarity

Quantifying Structure in CLIP Embeddings: A Statistical Framework for Concept Interpretation

2025-06-16 · Jitian Zhao, Chenghui Li, Frederic Sala, Karl Rohe

Concept-based approaches, which aim to identify human-understandable concepts within a model's internal representations, are a promising method for interpreting embeddings from deep neural network models, such as CLIP. W…

Measuring Semantic Coherence of a Conversation

2018-06-17 · Svitlana Vakulenko, Maarten de Rijke, Michael Cochez, Vadim Savenkov 외

Conversational systems have become increasingly popular as a way for humans to interact with computers. To be able to provide intelligent responses, conversational systems must correctly model the structure and semantics…

Knowledge Graphs