Modeling performance differences on cognitive tests using LSTMs and skip-thought vectors trained on reported media consumption.
Cognitive tests have traditionally resorted to standardizing testing materials in the name of equality and because of the onerous nature of creating test items. This approach ignores participants{'} diverse language experiences that potentially significantly affect testing outcomes. Here, we seek to explain our prior finding of significant performance differences on two cognitive tests (reading span and SPiN) between clusters of participants based on their media consumption. Here, we model the language contained in these media sources using an LSTM trained on corpora of each cluster{'}s media sources to predict target words. We also model semantic similarity of test items with each cluster{'}s corpus using skip-thought vectors. We find robust, significant correlations between performance on the SPiN test and the LSTMs and skip-thought models we present here, but not the reading span test.
Code (0)
등록된 구현이 없습니다.
Tasks
Semantic SimilaritySemantic Textual SimilaritySimilar Papers 제목 키워드 기반
Triangulating LLM Progress through Benchmarks, Games, and Cognitive Tests
We examine three evaluation paradigms: large question-answering benchmarks (e.g., MMLU and BBH), interactive games (e.g., Signalling Games or Taboo), and cognitive tests (e.g., for working memory or theory of mind). Firs…
Logical ReasoningMMLUQuestion AnsweringLanguage models emulate certain cognitive profiles: An investigation of how predictability measures interact with individual differences
To date, most investigations on surprisal and entropy effects in reading have been conducted on the group level, disregarding individual differences. In this work, we revisit the predictive power of surprisal and entropy…
Exploring Overall Contextual Information for Image Captioning in Human-Like Cognitive Style
Image captioning is a research hotspot where encoder-decoder models combining convolutional neural network (CNN) and long short-term memory (LSTM) achieve promising results. Despite significant progress, these models gen…
DecoderImage CaptioningSentenceCognitive Modeling of Semantic Fluency Using Transformers
Can deep language models be explanatory models of human cognition? If so, what are their limits? In order to explore this question, we propose an approach called hyperparameter hypothesization that uses predictive hyperp…
RetrievalValidity of Web-based, Self-directed, NeuroCognitive Performance Test in MCI
Digital cognitive tests offer several potential advantages over established paper-pencil tests but have not yet been fully evaluated for the clinical evaluation of mild cognitive impairment. The NeuroCognitive Performanc…
Skills Assessment