Sparse Autoencoder Features for Classifications and Transferability

2025-02-17 · Jack Gallifant, Shan Chen, Kuleen Sasse, Hugo Aerts, Thomas Hartvigsen, Danielle S. Bitterman

Sparse Autoencoders (SAEs) provide potentials for uncovering structured, human-interpretable representations in Large Language Models (LLMs), making them a crucial tool for transparent and controllable AI systems. We systematically analyze SAE for interpretable feature extraction from LLMs in safety-critical classification tasks. Our framework evaluates (1) model-layer selection and scaling properties, (2) SAE architectural configurations, including width and pooling strategies, and (3) the effect of binarizing continuous SAE activations. SAE-derived features achieve macro F1 > 0.8, outperforming hidden-state and BoW baselines while demonstrating cross-model transfer from Gemma 2 2B to 9B-IT models. These features generalize in a zero-shot manner to cross-lingual toxicity detection and visual classification tasks. Our analysis highlights the significant impact of pooling strategies and binarization thresholds, showing that binarization offers an efficient alternative to traditional feature selection while maintaining or improving performance. These findings establish new best practices for SAE-based interpretability and enable scalable, transparent deployment of LLMs in real-world applications. Full repo: https://github.com/shan23chen/MOSAIC.

📄 PDF Abstract BibTeX arXiv:2502.11367

Code (1)

shan23chen/mosaic 공식 구현 pytorch

Tasks

Binarizationfeature selection

Methods 이 논문이 사용한 방법론

Feature Selection Feature selection, also known as variable selection, attribute selection or variable subset selection, is the process of selecting a subset of relevant features (variables,…

Interpretable Company Similarity with Sparse Autoencoders

2024-12-03 · Marco Molinari, Victor Shao, Luca Imeneo, Mateusz Mikolajczak 외

Determining company similarity is a vital task in finance, underpinning risk management, hedging, and portfolio diversification. Practitioners often rely on sector and industry classifications such as SIC and GICS codes …

Large Language Model Semantic Similarity Semantic Textual Similarity

SAE as a Crystal Ball: Interpretable Features Predict Cross-domain Transferability of LLMs without Training

2026-03-03 · Qi Zhang, Yifei Wang, Xiaohan Wang, Jiajun Chai 외 arxiv

In recent years, pre-trained large language models have achieved remarkable success across diverse tasks. Besides the pivotal role of self-supervised pre-training, their effectiveness in downstream applications also depe…

Reinforcement Learning

Mechanistic Interpretability for Transformer-based Time Series Classification

2025-11-26 · Matīss Kalnāre, Sofoklis Kitharidis, Thomas Bäck, Niki van Stein arxiv

Transformer-based models have become state-of-the-art tools in various machine learning tasks, including time series classification, yet their complexity makes understanding their internal decision-making challenging. Ex…

Time Series Classification

How Pruning Reshapes Features: Sparse Autoencoder Analysis of Weight-Pruned Language Models

2026-03-26 · Hector Borobia, Elies Seguí-Mas, Guillermina Tormo-Carbó arxiv

Weight pruning is a standard technique for compressing large language models, yet its effect on learned internal representations remains poorly understood. We present the first systematic study of how unstructured prunin…

SAUCE: Selective Concept Unlearning in Vision-Language Models with Sparse Autoencoders

2025-03-16 · Qing Li, Jiahui Geng, Derui Zhu, Fengyu Cai 외

Unlearning methods for vision-language models (VLMs) have primarily adapted techniques from large language models (LLMs), relying on weight updates that demand extensive annotated forget sets. Moreover, these methods per…