Language Models Encode the Value of Numbers Linearly
Large language models (LLMs) have exhibited impressive competence in various tasks, but their internal mechanisms on mathematical problems are still under-explored. In this paper, we study a fundamental question: how language models encode the value of numbers, a basic element in math. To study the question, we construct a synthetic dataset comprising addition problems and utilize linear probes to read out input numbers from the hidden states. Experimental results support the existence of encoded number values in LLMs on different layers, and these values can be extracted via linear probes. Further experiments show that LLMs store their calculation results in a similar manner, and we can intervene the output via simple vector additions, proving the causal connection between encoded numbers and language model outputs. Our research provides evidence that LLMs encode the value of numbers linearly, offering insights for better exploring, designing, and utilizing numeric information in LLMs.
Code (1)
Tasks
Language ModelingLanguage ModellingMathSimilar Papers 제목 키워드 기반
Understanding the Countably Infinite: Neural Network Models of the Successor Function and its Acquisition
As children enter elementary school, their understanding of the ordinal structure of numbers transitions from a memorized count list of the first 50-100 numbers to knowing the successor function and understanding the cou…
Language Models Encode Numbers Using Digit Representations in Base 10
Large language models (LLMs) frequently make errors when handling even simple numerical problems, such as comparing two small numbers. A natural hypothesis is that these errors stem from how LLMs represent numbers, and s…
Probing for Multilingual Numerical Understanding in Transformer-Based Language Models
Natural language numbers are an example of compositional structures, where larger numbers are composed of operations on smaller numbers. Given that compositional reasoning is a key to natural language understanding, we p…
Natural Language UnderstandingNumber Representations in LLMs: A Computational Parallel to Human Perception
Humans are believed to perceive numbers on a logarithmic mental number line, where smaller values are represented with greater resolution than larger ones. This cognitive bias, supported by neuroscience and behavioral st…
Dimensionality ReductionMulti-Valued Cognitive Maps: Calculations with Linguistic Variables without Using Numbers
A concept of multi-valued cognitive maps is introduced in this paper. The concept expands the fuzzy one. However, all variables and weights are not linearly ordered in the concept, but are only partially-ordered. Such an…