Interpretability-Guided Layer Selection over Subspace Projection: SAEs as Stethoscopes, Not Scalpels, for Raw Task Vector Model Editing

2026-05-27 · Li Lei, Madalina Ciobanu, Qingqing Mao, Ritankar Das arxiv

LLMs increasingly require surgical model editing to enhance domain-specific capabilities without incurring the computational cost or catastrophic forgetting associated with full fine-tuning. Sparse Autoencoders (SAEs) have emerged as a promising tool in this setting, in principle allowing for feature-level identification of where to intervene. In this work, we rigorously evaluate an SAE-guided editing pipeline for mathematical reasoning on Gemma-3-4B-IT and uncover a fundamental failure mode: the intuitively appealing approach of projecting task vectors onto SAE feature subspaces acts as an information bottleneck that discards approximately 97% of the modification energy, yielding no statistically significant improvements across seven math subjects. We show that this failure stems from a geometric misalignment between activation-space SAE directions and weight-space task vectors. We then propose a shift in perspective: SAE as a Stethoscope, Not a Scalpel, where SAEs are used for layer-level diagnosis rather than intervention-level filtering. By injecting unfiltered raw task vectors only into layers identified by an SAE-derived specificity score, we improve Number Theory accuracy from 29.6% to 39.4% (z=+3.41, p=0.0007) on the Minerva Math benchmark; 5 of 7 math subjects significantly improved and none significantly degraded. Our method is fully deterministic, requires no additional inference cost, and provides a principled framework for interpretability-guided model editing.

📄 PDF Abstract BibTeX arXiv:2605.28649

Code (0)

등록된 구현이 없습니다.

Tasks

Mathematical Reasoning

Model Unlearning via Sparse Autoencoder Subspace Guided Projections

2025-05-30 · Xu Wang, Zihao Li, Benyou Wang, Yan Hu 외

Large language models (LLMs) store vast amounts of information, making them powerful yet raising privacy and safety concerns when selective knowledge removal is required. Existing unlearning strategies, ranging from grad…

Adversarial Robustness feature selection GSM8K MMLU+2

Layer-wise Representation Dynamics: An Empirical Investigation Across Embedders and Base LLMs

2026-05-12 · Jingzhou Jiang, Yi Yang, Kar Yan Tam arxiv

Hidden states change substantially across the layers of modern language models, but most layer-wise analyses focus on one aspect of that change. We propose Layer-wise Representation Dynamics (LRD), a framework with three…

A Subspace-based Approach for Dimensionality Reduction and Important Variable Selection

2021-06-03 · Di Bo, Hoon Hwangbo, Vinit Sharma, Corey Arndt 외

An analysis of high-dimensional data can offer a detailed description of a system but is often challenged by the curse of dimensionality. General dimensionality reduction techniques can alleviate such difficulty by extra…

Decision Making Dimensionality Reduction Variable Selection

Segmentation of Cardiac Structures via Successive Subspace Learning with Saab Transform from Cine MRI

2021-07-22 · Xiaofeng Liu, Fangxu Xing, Hanna K. Gaggin, Weichung Wang 외

Assessment of cardiovascular disease (CVD) with cine magnetic resonance imaging (MRI) has been used to non-invasively evaluate detailed cardiac structure and function. Accurate segmentation of cardiac structures from cin…

Dimensionality Reduction feature selection Interpretable Machine Learning Prognosis+1

Uncovering the Latent Potential of Deep Intermediate Representations

2026-05-21 · Arnesh Batra, Arush Gumber, Aniket Khandelwal, Jashn Khemani 외 arxiv

Foundational Models pretrained on huge amount of data learn representations that evolve across depth, forming a hierarchy of embeddings with distinct semantic content and geometric structure. Contrary to the widespread p…