Learning to Explain: An Information-Theoretic Perspective on Model Interpretation
We introduce instancewise feature selection as a methodology for model interpretation. Our method is based on learning a function to extract a subset of features that are most informative for each given example. This feature selector is trained to maximize the mutual information between selected features and the response variable, where the conditional distribution of the response variable given the input is the model to be explained. We develop an efficient variational approximation to the mutual information, and show the effectiveness of our method on a variety of synthetic and real data sets using both quantitative metrics and human evaluation.
Code (3)
Tasks
feature selectionSimilar Papers 제목 키워드 기반
MED-TEX: Transferring and Explaining Knowledge with Less Data from Pretrained Medical Imaging Models
Deep learning methods usually require a large amount of training data and lack interpretability. In this paper, we propose a novel knowledge distillation and model interpretation framework for medical image classificatio…
image-classificationImage ClassificationKnowledge DistillationMedical Image ClassificationRethinking GSPO: The Perplexity-Entropy Equivalence
We provide a new perspective on GSPO's length-normalized importance ratios by establishing their connection to information-theoretic quantities. We show that GSPO's sequence-level weight $s(θ) = (π_θ/π_{θ_{\text{old}}})^…
Mathematical ReasoningA Mathematical Explanation of Transformers
The Transformer architecture has revolutionized the field of sequence modeling and underpins the recent breakthroughs in large language models (LLMs). However, a comprehensive mathematical theory that explains its struct…
Benchmarking Machine Reading Comprehension: A Psychological Perspective
Machine reading comprehension (MRC) has received considerable attention as a benchmark for natural language understanding. However, the conventional task design of MRC lacks explainability beyond the model interpretation…
BenchmarkingMachine Reading ComprehensionNatural Language UnderstandingPosition+1Information Theoretic Interpretation of Deep learning
We interpret part of the experimental results of Shwartz-Ziv and Tishby [2017]. Inspired by these results, we established a conjecture of the dynamics of the machinary of deep neural network. This conjecture can be used …
Deep Learning