Understanding and Mitigating Dataset Corruption in LLM Steering
Contrastive steering has been shown as a simple and effective method to adjust the generative behavior of LLMs at inference time. It uses examples of prompt responses with and without a trait to identify a direction in an intermediate activation layer, and then shifts activations in this 1-dimensional subspace. However, despite its growing use in AI safety applications, the robustness of contrastive steering to noisy or adversarial data corruption is poorly understood. We initiate a study of the robustness of this process with respect to corruption of the dataset of examples used to train the steering direction. Our first observation is that contrastive steering is quite robust to a moderate amount of corruption, but unwanted side effects can be clearly and maliciously manifested when a non-trivial fraction of the training data is altered. Second, we analyze the geometry of various types of corruption, and identify some safeguards. Notably, a key step in learning the steering direction involves high-dimensional mean computation, and we show that replacing this step with a recently developed robust mean estimator often mitigates most of the unwanted effects of malicious corruption.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Gradient-Free Adversarial Training Against Image Corruption for Learning-based Steering
We introduce a simple yet effective framework for improving the robustness of learning algorithms against image corruptions for autonomous driving. These corruptions can occur due to both internal (e.g., sensor noises an…
Autonomous DrivingSelf-Driving CarsSAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucination
Although Multimodal Large Language Models (MLLMs) have advanced substantially, they remain vulnerable to object hallucination caused by language priors and visual information loss. To address this, we propose SAVE (Spars…
A Fourier Perspective on Model Robustness in Computer Vision
Achieving robustness to distributional shift is a longstanding and challenging goal of computer vision. Data augmentation is a commonly used approach for improving robustness, however robustness gains are typically not u…
Data AugmentationSteeringTTA: Guiding Diffusion Trajectories for Robust Test-Time-Adaptation
Test-time adaptation (TTA) aims to correct performance degradation of deep models under distribution shifts by updating models or inputs using unlabeled test data. Input-only diffusion-based TTA methods improve robustnes…
Test-time AdaptationMitigating Content Effects on Reasoning in Language Models through Fine-Grained Activation Steering
Large language models (LLMs) frequently demonstrate reasoning limitations, often conflating content plausibility (i.e., material inference) with logical validity (i.e., formal inference). This can result in biased infere…
Language ModelingLanguage Modelling