Long-range and hierarchical language predictions in brains and algorithms
Deep learning has recently made remarkable progress in natural language processing. Yet, the resulting algorithms remain far from competing with the language abilities of the human brain. Predictive coding theory offers a potential explanation to this discrepancy: while deep language algorithms are optimized to predict adjacent words, the human brain would be tuned to make long-range and hierarchical predictions. To test this hypothesis, we analyze the fMRI brain signals of 304 subjects each listening to 70min of short stories. After confirming that the activations of deep language algorithms linearly map onto those of the brain, we show that enhancing these models with long-range forecast representations improves their brain-mapping. The results further reveal a hierarchy of predictions in the brain, whereby the fronto-parietal cortices forecast more abstract and more distant representations than the temporal cortices. Overall, this study strengthens predictive coding theory and suggests a critical role of long-range and hierarchical predictions in natural language processing.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
The Thousand Brains Project: A New Paradigm for Sensorimotor Intelligence
Artificial intelligence has advanced rapidly in the last decade, driven primarily by progress in the scale of deep-learning systems. Despite these advances, the creation of intelligent systems that can operate effectivel…
Continual LearningHierarchy or Heterarchy? A Theory of Long-Range Connections for the Sensorimotor Brain
In the traditional understanding of the neocortex, sensory information flows up a hierarchy of regions, with each level processing increasingly complex features. Information also flows down the hierarchy via a different …
BrainSegNet : A Segmentation Network for Human Brain Fiber Tractography Data into Anatomically Meaningful Clusters
The segregation of brain fiber tractography data into distinct and anatomically meaningful clusters can help to comprehend the complex brain structure and early investigation and management of various neural disorders. W…
ClassificationGeneral ClassificationManagementBrainstorming Brings Power to Large Language Models of Knowledge Reasoning
Large Language Models (LLMs) have demonstrated amazing capabilities in language generation, text comprehension, and knowledge reasoning. While a single powerful model can already handle multiple tasks, relying on a singl…
Logical ReasoningReading ComprehensionText GenerationVISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping
In argumentative writing, writers must brainstorm hierarchical writing goals, ensure the persuasiveness of their arguments, and revise and organize their plans through drafting. Recent advances in large language models (…
PersuasivenessText Generation