Towards Reasonable Concept Bottleneck Models
In this paper, we propose $\textbf{C}$oncept $\textbf{REA}$soning $\textbf{M}$odels (CREAM), a novel family of Concept Bottleneck Models (CBMs) that: (i) explicitly encodes concept-concept (${\texttt{C-C}}$) and concept-task (${\texttt{C$\rightarrow$Y}}$) relationships to enforce a desired model reasoning; and (ii) use a regularized side-channel to achieve competitive task performance, while keeping high concept importance. Specifically, CREAM architecturally embeds (bi)directed concept-concept, and concept to task relationships specified by a human expert, while severing undesired information flows (e.g., to handle mutually exclusive concepts). Moreover, CREAM integrates a black-box side-channel that is regularized to encourage task predictions to be grounded in the relevant concepts, thereby utilizing the side-channel only when necessary to enhance performance. Our experiments show that: (i) CREAM mainly relies on concepts while achieving task performance on par with black-box models; and (ii) the embedded ${\texttt{C-C}}$ and ${\texttt{C$\rightarrow$Y}}$ relationships ease model interventions and mitigate concept leakage.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A General Framework for a Class of Quarrels: The Quarrelling Paradox Revisited
If a measure of voting power assigns greater voting power to a player because it no longer effectively cooperates with another, then the measure displays the quarrelling paradox and violates the quarrel postulate. We pro…
Do Concept Bottleneck Models Learn as Intended?
Concept bottleneck models map from raw inputs to concepts, and then from concepts to targets. Such models aim to incorporate pre-specified, high-level concepts into the learning procedure, and have been motivated to meet…
CoBELa: Steering Transparent Generation via Concept Bottlenecks on Energy Landscapes
Generative concept bottleneck models aim to enable interpretable generation by routing synthesis through explicit, user-facing concepts. In practice, prior approaches often rely on non-explicit bottleneck representations…
Towards Achieving Concept Completeness for Textual Concept Bottleneck Models
Textual Concept Bottleneck Models (TBMs) are interpretable-by-design models for text classification that predict a set of salient concepts before making the final prediction. This paper proposes Complete Textual Concept …
Language ModelingLanguage ModellingSmall Language Modeltext-classification+1There Was Never a Bottleneck in Concept Bottleneck Models
Deep learning representations are often difficult to interpret, which can hinder their deployment in sensitive applications. Concept Bottleneck Models (CBMs) have emerged as a promising approach to mitigate this issue by…