Causal Graph in Language Model Rediscovers Cortical Hierarchy in Human Narrative Processing
Understanding how humans process natural language has long been a vital research direction. The field of natural language processing (NLP) has recently experienced a surge in the development of powerful language models. These models have proven to be invaluable tools for studying another complex system known to process human language: the brain. Previous studies have demonstrated that the features of language models can be mapped to fMRI brain activity. This raises the question: is there a commonality between information processing in language models and the human brain? To estimate information flow patterns in a language model, we examined the causal relationships between different layers. Drawing inspiration from the workspace framework for consciousness, we hypothesized that features integrating more information would more accurately predict higher hierarchical brain activity. To validate this hypothesis, we classified language model features into two categories based on causal network measures: 'low in-degree' and 'high in-degree'. We subsequently compared the brain prediction accuracy maps for these two groups. Our results reveal that the difference in prediction accuracy follows a hierarchical pattern, consistent with the cortical hierarchy map revealed by activity time constants. This finding suggests a parallel between how language models and the human brain process linguistic information.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingSimilar Papers 제목 키워드 기반
Preterm neonates distinguish rhythm violation through a hierarchy of cortical processing
Rhythm is a fundamental component of the auditory world, present even during the prenatal life. While there is evidence that some auditory capacities are already present before birth, whether and how the premature neural…
RhythmIntegrating Causal Inference with Graph Neural Networks for Alzheimer's Disease Analysis
Deep graph learning has advanced Alzheimer's (AD) disease classification from MRI, but most models remain correlational, confounding demographic and genetic factors with disease specific features. We present Causal-GCN, …
Causal InferenceGraph LearningA High-Level Model of Neocortical Feedback Based on an Event Window Segmentation Algorithm
The author previously presented an event window segmentation (EWS) algorithm [5] that uses purely statistical methods to learn to recognize recurring patterns in an input stream of events. In the following discussion, th…
UNMASK: Discovering and Causally Verifying Spurious Shortcuts in Text Classifiers
Neural language models trained on large crowdsourced corpora frequently exploit spurious surface patterns tied to target labels without true linguistic or causal relevance, boosting benchmark performance while failing on…
ReBeCA: Unveiling Interpretable Behavior Hierarchy behind the Iterative Self-Reflection of Language Models with Causal Analysis
While self-reflection can enhance language model reliability, its underlying mechanisms remain opaque, with existing analyses often yielding correlation-based insights that fail to generalize. To address this, we introdu…