Exploring Task Performance with Interpretable Models via Sparse Auto-Encoders

2025-07-08 · Shun Wang, Tyler Loakman, Youbo Lei, Yi Liu, Bohao Yang, Yuting Zhao, Dong Yang, Chenghua Lin arxiv

Large Language Models (LLMs) are traditionally viewed as black-box algorithms, therefore reducing trustworthiness and obscuring potential approaches to increasing performance on downstream tasks. In this work, we apply an effective LLM decomposition method using a dictionary-learning approach with sparse autoencoders. This helps extract monosemantic features from polysemantic LLM neurons. Remarkably, our work identifies model-internal misunderstanding, allowing the automatic reformulation of the prompts with additional annotations to improve the interpretation by LLMs. Moreover, this approach demonstrates a significant performance improvement in downstream tasks, such as mathematical reasoning and metaphor detection.

📄 PDF Abstract BibTeX arXiv:2507.06427

Code (0)

등록된 구현이 없습니다.

Tasks

Mathematical Reasoning

Disentangling Dense Embeddings with Sparse Autoencoders

2024-08-01 · Charles O'Neill, Christine Ye, Kartheik Iyer, John F. Wu

Sparse autoencoders (SAEs) have shown promise in extracting interpretable features from complex neural networks. We present one of the first applications of SAEs to dense text embeddings from large language models, demon…

Astronomy

Less is Enough: Synthesizing Diverse Data in LLM Feature Space with Sparse Autoencoders

2026-02-11 · Zhongzhi Li, Xuansheng Wu, Yijiang Li, Lijie Hu 외 arxiv

The diversity of post-training data is critical for effective downstream performance in large language models (LLMs). Many existing approaches to constructing post-training data quantify diversity using text-based metric…

Instruction Following

DeepSynth: Automata Synthesis for Automatic Task Segmentation in Deep Reinforcement Learning

2019-11-22 · Mohammadhosein Hasanbeig, Natasha Yogananda Jeppu, Alessandro Abate, Tom Melham 외

This paper proposes DeepSynth, a method for effective training of deep Reinforcement Learning (RL) agents when the reward is sparse and non-Markovian, but at the same time progress towards the reward requires achieving a…

Deep Reinforcement Learning Hierarchical Reinforcement Learning Montezuma's Revenge Probabilistic Deep Learning+4

Interpreting Attention Layer Outputs with Sparse Autoencoders

2024-06-25 · Connor Kissane, Robert Krzyzanowski, Joseph Isaac Bloom, Arthur Conmy 외

Decomposing model activations into interpretable components is a key open problem in mechanistic interpretability. Sparse autoencoders (SAEs) are a popular method for decomposing the internal activations of trained trans…

SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentation

2026-05-21 · Zhenyu Lu, Liupeng Li, Jinpeng Wang, Haoqian Kang 외 arxiv

While large language models provide strong compositional reasoning, existing reasoning segmentation pipelines fail to transparently connect this reasoning to visual perception. Current methods, such as latent query align…

Reinforcement Learning