Text Understanding in GPT-4 vs Humans
We examine whether a leading AI system GPT4 understands text as well as humans do, first using a well-established standardized test of discourse comprehension. On this test, GPT4 performs slightly, but not statistically significantly, better than humans given the very high level of human performance. Both GPT4 and humans make correct inferences about information that is not explicitly stated in the text, a critical test of understanding. Next, we use more difficult passages to determine whether that could allow larger differences between GPT4 and humans. GPT4 does considerably better on this more difficult text than do the high school and university students for whom these the text passages are designed, as admission tests of student reading comprehension. Deeper exploration of GPT4 performance on material from one of these admission tests reveals generally accepted signatures of genuine understanding, namely generalization and inference.
Code (0)
등록된 구현이 없습니다.
Tasks
Reading ComprehensionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Humans Perceive Wrong Narratives from AI Reasoning Texts
A new generation of AI models generates step-by-step reasoning text before producing an answer. This text appears to offer a human-readable window into their computation process, and is increasingly relied upon for trans…
ProSPer: Probing Human and Neural Network Language Model Understanding of Spatial Perspective
Understanding perspectival language is important for applications like dialogue systems and human-robot interaction. We propose a probe task that explores how well language models understand spatial perspective. We prese…
Language ModelingLanguage ModellingCognitive Perspectives on Context-based Decisions and Explanations
When human cognition is modeled in Philosophy and Cognitive Science, there is a pervasive idea that humans employ mental representations in order to navigate the world and make predictions about outcomes of future action…
Decision MakingExplainable Artificial Intelligence (XAI)NavigatePhilosophyHumanSense: From Multimodal Perception to Empathetic Context-Aware Responses through Reasoning MLLMs
While Multimodal Large Language Models (MLLMs) show immense promise for achieving truly human-like interactions, progress is hindered by the lack of fine-grained evaluation frameworks for human-centered scenarios, encomp…
Reinforcement LearningDissonance Between Human and Machine Understanding
Complex machine learning models are deployed in several critical domains including healthcare and autonomous vehicles nowadays, albeit as functional black boxes. Consequently, there has been a recent surge in interpretin…
AttributeAutonomous Vehiclesfeature selectionimage-classification+1