DePass: Unified Feature Attributing by Simple Decomposed Forward Pass
Attributing the behavior of Transformer models to internal computations is a central challenge in mechanistic interpretability. We introduce DePass, a unified framework for feature attribution based on a single decomposed forward pass. DePass decomposes hidden states into customized additive components, then propagates them with attention scores and MLP's activations fixed. It achieves faithful, fine-grained attribution without requiring auxiliary training. We validate DePass across token-level, model component-level, and subspace-level attribution tasks, demonstrating its effectiveness and fidelity. Our experiments highlight its potential to attribute information flow between arbitrary components of a Transformer model. We hope DePass serves as a foundational tool for broader applications in interpretability.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Decomposition Ascribed Synergistic Learning for Unified Image Restoration
Learning to restore multiple image degradations within a single model is quite beneficial for real-world applications. Nevertheless, existing works typically concentrate on regarding each degradation independently, while…
DeblurringDenoisingImage DeblurringImage Dehazing+6Make Your Decision Convincing! A Unified Two-Stage Framework: Self-Attribution and Decision-Making
Explaining black-box model behavior with natural language has achieved impressive results in various NLP tasks. Recent research has explored the utilization of subsequences from the input text as a rationale, providing u…
Decision MakingUnified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systems
Learning high-quality feature embeddings efficiently and effectively is critical for the performance of web-scale machine learning systems. A typical model ingests hundreds of features with vocabularies on the order of m…
Recommendation SystemsA Characteristic Function for Shapley-Value-Based Attribution of Anomaly Scores
In anomaly detection, the degree of irregularity is often summarized as a real-valued anomaly score. We address the problem of attributing such anomaly scores to input features for interpreting the results of anomaly det…
Anomaly DetectionSemi-supervised Anomaly DetectionSupervised Anomaly DetectionMeasurements of Three-Level Hierarchical Structure in the Outliers in the Spectrum of Deepnet Hessians
We consider deep classifying neural networks. We expose a structure in the derivative of the logits with respect to the parameters of the model, which is used to explain the existence of outliers in the spectrum of the H…