Promises and Pitfalls of Black-Box Concept Learning Models
Machine learning models that incorporate concept learning as an intermediate step in their decision making process can match the performance of black-box predictive models while retaining the ability to explain outcomes in human understandable terms. However, we demonstrate that the concept representations learned by these models encode information beyond the pre-defined concepts, and that natural mitigation strategies do not fully work, rendering the interpretation of the downstream prediction misleading. We describe the mechanism underlying the information leakage and suggest recourse for mitigating its effects.
Code (2)
Tasks
Decision MakingSimilar Papers 제목 키워드 기반
Promises and pitfalls of deep neural networks in neuroimaging-based psychiatric research
By promising more accurate diagnostics and individual treatment recommendations, deep neural networks and in particular convolutional neural networks have advanced to a powerful tool in medical imaging. Here, we first gi…
Transfer LearningNatural Language Processing for Drug Discovery Knowledge Graphs: promises and pitfalls
Building and analysing knowledge graphs (KGs) to aid drug discovery is a topical area of research. A salient feature of KGs is their ability to combine many heterogeneous data sources in a format that facilitates discove…
Drug DiscoveryKnowledge Graphsnamed-entity-recognitionNamed Entity RecognitionResponse to Promises and Pitfalls of Deep Kernel Learning
This note responds to "Promises and Pitfalls of Deep Kernel Learning" (Ober et al., 2021). The marginal likelihood of a Gaussian process can be compartmentalized into a data fit term and a complexity penalty. Ober et al.…
Words as Bridges: Exploring Computational Support for Cross-Disciplinary Translation Work
Scholars often explore literature outside of their home community of study. This exploration process is frequently hampered by field-specific jargon. Past computational work often focuses on supporting translation work b…
TranslationWord EmbeddingsA Survey on Graph Counterfactual Explanations: Definitions, Methods, Evaluation, and Research Challenges
Graph Neural Networks (GNNs) perform well in community detection and molecule classification. Counterfactual Explanations (CE) provide counter-examples to overcome the transparency limitations of black-box models. Due to…
BenchmarkingCommunity DetectioncounterfactualCounterfactual Explanation+3