Attention Sinks and Outlier Features: A 'Catch, Tag, and Release' Mechanism for Embeddings

2025-02-02 · Stephen Zhang, Mustafa Khan, Vardan Papyan

Two prominent features of large language models (LLMs) is the presence of large-norm (outlier) features and the tendency for tokens to attend very strongly to a select few tokens. Despite often having no semantic relevance, these select tokens, called attention sinks, along with the large outlier features, have proven important for model performance, compression, and streaming. Consequently, investigating the roles of these phenomena within models and exploring how they might manifest in the model parameters has become an area of active interest. Through an empirical investigation, we demonstrate that attention sinks utilize outlier features to: catch a sequence of tokens, tag the captured tokens by applying a common perturbation, and then release the tokens back into the residual stream, where the tagged tokens are eventually retrieved. We prove that simple tasks, like averaging, necessitate the 'catch, tag, release' mechanism hence explaining why it would arise organically in modern LLMs. Our experiments also show that the creation of attention sinks can be completely captured in the model parameters using low-rank matrices, which has important implications for model compression and substantiates the success of recent approaches that incorporate a low-rank term to offset performance degradation.

📄 PDF Abstract BibTeX arXiv:2502.00919

Code (0)

등록된 구현이 없습니다.

Tasks

Model CompressionTAG

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Attention 설명 없음

Attention Sinks Please enter a description about the method here

A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training

2026-01-30 · Zihan Qiu, Zeyu Huang, Kaiyue Wen, Peng Jin 외 arxiv

We investigate the functional role of emergent outliers in large language models, specifically attention sinks (a few tokens that consistently receive large attention logits) and residual sinks (a few fixed dimensions wi…

KVSink: Understanding and Enhancing the Preservation of Attention Sinks in KV Cache Quantization for LLMs

2025-08-06 · Zunhai Su, Kehong Yuan arxiv

Key-Value (KV) cache quantization has become a widely adopted optimization technique for efficient large language models (LLMs) inference by reducing KV cache memory usage and mitigating memory-bound constraints. Recent …

Rotary Outliers and Rotary Offset Features in Large Language Models

2025-03-03 · André Jonasson

Transformer-based Large Language Models (LLMs) rely on positional encodings to provide sequence position information to their attention mechanism. Rotary Positional Encodings (RoPE), which encode relative position by rot…

Position

Attention Sinks and Outliers in Attention Residuals

2026-05-18 · Haozheng Luo, Haoran Dai, Shaoyang Zhang, Xi Chen 외 arxiv

We propose OASIS, an outlier- and sink-aware technique built on inter-layer null signaling. As AttnResidual architectures introduce an additional depth-wise normalization channel, they improve inter-layer routing flexibi…

The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks

2026-03-05 · Shangwen Sun, Alfredo Canziani, Yann LeCun, Jiachen Zhu arxiv

We study two recurring phenomena in Transformer language models: massive activations, in which a small number of tokens exhibit extreme outliers in a few channels, and attention sinks, in which certain tokens attract dis…