Before the Last Token: Diagnosing Final-Token Safety Probe Failures

2026-05-12 · Shravan Doda arxiv

Final-token safety probes monitor a single hidden state after prompt prefill, but jailbreak prompts can contain probe-visible unsafe evidence distributed across earlier user-token representations that is missed by this readout. We study this prefill-time failure mode using SafeSwitch-style probes trained only on clean harmful and benign prompts across three instruction-tuned LLMs. The probes achieve high recall on clean harmful prompts, but miss many jailbreaks and can produce false positives on safety-adjacent benign prompts. Subspace analyses suggest that missed jailbreaks differ from clean benign prompts along directions that are poorly captured by the probe's representational subspace, and increasing probe bottleneck width does not reliably resolve this mismatch. Token-level prefill analyses reveal that probe-visible unsafe evidence often appears earlier in the sequence but is not exposed at the final-token readout, while naive max-pooling over token positions overfires on safe prompts. A simple PCA-HMM trajectory model, trained only on the same clean split, recovers many final-token misses from user-content prefill trajectories without the catastrophic false-positive behavior of naive token pooling, motivating trajectory-aware hidden-state analyses as diagnostic complements to final-token probes

📄 PDF Abstract BibTeX arXiv:2605.12726

Code (0)

등록된 구현이 없습니다.

LAST: The Last Query Token Guides Visual Token Pruning for Edge-Cloud Collaborative MLLM Inference

2026-07-30 · Feng Yang, Xinrui Ju, Keyang Zhang, Xiandong Meng 외 arxiv

Multimodal foundation models are reshaping edge-cloud visual intelligence from task-specific feature pipelines into token-based interfaces, where edge devices encode visual inputs into tokens for a general-purpose cloud …

Diagnosing Memorization in Chain-of-Thought Reasoning, One Token at a Time

2025-08-04 · Huihan Li, You Chen, Siyuan Wang, Yixin He 외 arxiv

Large Language Models (LLMs) perform well on reasoning benchmarks but often fail when inputs alter slightly, raising concerns about the extent to which their success relies on memorization. This issue is especially acute…

jina-reranker-v3: Last but Not Late Interaction for Listwise Document Reranking

2025-09-29 · Feng Wang, Yuqing Li, Han Xiao arxiv

jina-reranker-v3 is a 0.6B-parameter multilingual listwise reranker that introduces a novel "last but not late" interaction. Unlike late interaction models like ColBERT that encode documents separately before multi-vecto…

Think before you speak: Training Language Models With Pause Tokens

2023-10-03 · Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon 외

Language models generate responses by producing a series of tokens in immediate succession: the $(K+1)^{th}$ token is an outcome of manipulating $K$ hidden vectors per layer, one vector per preceding token. What if inste…

Decoder GSM8K Question Answering

Towards Implicit Aggregation: Robust Image Representation for Place Recognition in the Transformer Era

2025-11-08 · Feng Lu, Tong Jin, Canming Ye, Yunpeng Liu 외 arxiv

Visual place recognition (VPR) is typically regarded as a specific image retrieval task, whose core lies in representing images as global descriptors. Over the past decade, dominant VPR methods (e.g., NetVLAD) have follo…

Visual Place Recognition Image Retrieval