Steering Code LLMs with Activation Directions for Language and Library Control
Code LLMs often default to particular programming languages and libraries under neutral prompts. We investigate whether these preferences are encoded as approximately linear directions in activation space that can be manipulated at inference time. Using a difference-in-means method, we estimate layer-wise steering vectors for five language/library pairs and add them to model hidden states during generation. Across three open-weight code LLMs, these interventions substantially increase generation toward the target ecosystem under neutral prompts and often remain effective even when prompts explicitly request the opposite choice. Steering strength varies by model and target, with common ecosystems easier to induce than rarer alternatives, and overly strong interventions can reduce output quality. Overall, our results suggest that code-style preferences in LLMs are partly represented by compact, steerable structure in activation space.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steering
Large language models (LLMs) are prone to capturing biases from training corpus, leading to potential negative social impacts. Existing prompt-based debiasing methods exhibit instability due to their sensitivity to promp…
counterfactualFairnessQuestion AnsweringText GenerationDo LLMs Experience an Internal Polylogue? Investigating Reasoning through the Lens of Personas
Recent work shows that large language models (LLMs) encode behavioural traits ("personas") as linear directions in activation space, often called "persona vectors". Prior work has used such directions as static handles f…
Neural FOXP2 -- Language Specific Neuron Steering for Targeted Language Improvement in LLMs
LLMs are multilingual by training, yet their lingua franca is often English, reflecting English language dominance in pretraining. Other languages remain in parametric memory but are systematically suppressed. We argue t…
Activation Steering with a Feedback Controller
Controlling the behaviors of large language models (LLM) is fundamental to their safety alignment and reliable deployment. However, existing steering methods are primarily driven by empirical insights and lack theoretica…
Linear Personality Probing and Steering in LLMs: A Big Five Study
Large language models (LLMs) exhibit distinct and consistent personalities that greatly impact trust and engagement. While this means that personality frameworks would be highly valuable tools to characterize and control…
Prompt Engineering