Concepts' Information Bottleneck Models
Concept Bottleneck Models (CBMs) aim to deliver interpretable predictions by routing decisions through a human-understandable concept layer, yet they often suffer reduced accuracy and concept leakage that undermines faithfulness. We introduce an explicit Information Bottleneck regularizer on the concept layer that penalizes $I(X;C)$ while preserving task-relevant information in $I(C;Y)$, encouraging minimal-sufficient concept representations. We derive two practical variants (a variational objective and an entropy-based surrogate) and integrate them into standard CBM training without architectural changes or additional supervision. Evaluated across six CBM families and three benchmarks, the IB-regularized models consistently outperform their vanilla counterparts. Information-plane analyses further corroborate the intended behavior. These results indicate that enforcing a minimal-sufficient concept bottleneck improves both predictive performance and the reliability of concept-level interventions. The proposed regularizer offers a theoretic-grounded, architecture-agnostic path to more faithful and intervenable CBMs, resolving prior evaluation inconsistencies by aligning training protocols and demonstrating robust gains across model families and datasets.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Learning Concept Bottleneck Models from Mechanistic Explanations
Concept Bottleneck Models (CBMs) aim for ante-hoc interpretability by learning a bottleneck layer that predicts interpretable concepts before the decision. State-of-the-art approaches typically select which concepts to l…
Knowledge GraphsLanguage in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification
Concept Bottleneck Models (CBM) are inherently interpretable models that factor model decisions into human-readable concepts. They allow people to easily understand why a model is failing, a critical feature for high-sta…
image-classificationImage ClassificationLanguage ModelingLanguage ModellingEliminating Information Leakage in Hard Concept Bottleneck Models with Supervised, Hierarchical Concept Learning
Concept Bottleneck Models (CBMs) aim to deliver interpretable and interventionable predictions by bridging features and labels with human-understandable concepts. While recent CBMs show promising potential, they suffer f…
The Decoupling Concept Bottleneck Model
The Concept Bottleneck Model (CBM) is an interpretable neural network that leverages high-level concepts to explainmodel decisions and conduct human-machine interaction. However, in real-world scenarios, the deficiency o…
modelMutual Information EstimationTowards learning to explain with concept bottleneck models: mitigating information leakage
Concept bottleneck models perform classification by first predicting which of a list of human provided concepts are true about a datapoint. Then a downstream model uses these predicted concept labels to predict the targe…