Human Sentence Processing: Recurrence or Attention?
Recurrent neural networks (RNNs) have long been an architecture of interest for computational models of human sentence processing. The recently introduced Transformer architecture outperforms RNNs on many natural language processing tasks but little is known about its ability to model human language processing. We compare Transformer- and RNN-based language models' ability to account for measures of human reading effort. Our analysis shows Transformers to outperform RNNs in explaining self-paced reading times and neural activity during reading English sentences, challenging the widely held idea that human sentence processing involves recurrent and immediate processing and provides evidence for cue-based retrieval.
Code (1)
Tasks
Language ModellingRetrievalSentenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Staircase Attention for Recurrent Processing of Sequences
Attention mechanisms have become a standard tool for sequence modeling tasks, in particular by stacking self-attention layers over the entire input sequence as in the Transformer architecture. In this work we introduce a…
Language ModelingLanguage ModellingSplit-and-Rephrase in a Cross-Lingual Manner: A Complete Pipeline
Split-and-rephrase is a challenging task that promotes the transformation of a given complex input sentence into multiple shorter sentences retaining equivalent meaning. This rewriting approach conceptualizes that shorte…
Language ModelingLanguage ModellingMasked Language ModelingPOS+2A Language Model with Limited Memory Capacity Captures Interference in Human Sentence Processing
Two of the central factors believed to underpin human sentence processing difficulty are expectations and retrieval from working memory. A recent attempt to create a unified cognitive model integrating these two factors …
Language ModelingLanguage ModellingRetrievalSentenceDeep Learning for End-to-End Atrial Fibrillation Recurrence Estimation
Left atrium shape has been shown to be an independent predictor of recurrence after atrial fibrillation (AF) ablation. Shape-based representation is imperative to such an estimation process, where correspondence-based re…
AnatomyAtrial Fibrillation Recurrence EstimationData AugmentationDeep Learning+3If Attention Serves as a Cognitive Model of Human Memory Retrieval, What is the Plausible Memory Representation?
Recent work in computational psycholinguistics has revealed intriguing parallels between attention mechanisms and human memory retrieval, focusing primarily on Transformer architectures that operate on token-level repres…
RetrievalSentence