Beyond Turing Test: Can GPT-4 Sway Experts' Decisions?
In the post-Turing era, evaluating large language models (LLMs) involves assessing generated text based on readers' reactions rather than merely its indistinguishability from human-produced content. This paper explores how LLM-generated text impacts readers' decisions, focusing on both amateur and expert audiences. Our findings indicate that GPT-4 can generate persuasive analyses affecting the decisions of both amateurs and professionals. Furthermore, we evaluate the generated text from the aspects of grammar, convincingness, logical coherence, and usefulness. The results highlight a high correlation between real-world evaluation through audience reactions and the current multi-dimensional evaluators commonly used for generative models. Overall, this paper shows the potential and risk of using generated text to sway human decisions and also points out a new direction for evaluating generated text, i.e., leveraging the reactions and decisions of readers. We release our dataset to assist future research.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Assessing the Reliability and Validity of a Balance Mat for Measuring Postural Stability: A Combined Robot-Human Approach
Postural sway assessment is important for detecting balance problems and identifying people at risk of falls. Force plates (FP) are considered the gold standard postural sway assessment method in laboratory conditions, b…
Body-worn triaxial accelerometer coherence and reliability related to static posturography in unilateral vestibular failure
Due to the fact that no study to date has shown the experimental validity of ACC-based measures of body sway with respect to posturography for subjects with vestibular deficits, the aim of the present study was: i) to de…
The expressway network design problem for multiple urban subregions based on the macroscopic fundamental diagram
As urbanization advances, cities are expanding, leading to a more decentralized urban structure and longer average commuting durations. The construction of an urban expressway system emerges as a critical strategy to tac…
Agentsway -- Software Development Methodology for AI Agents-based Teams
The emergence of Agentic AI is fundamentally transforming how software is designed, developed, and maintained. Traditional software development methodologies such as Agile, Kanban, ShapeUp, etc, were originally designed …
Monocular 3D Sway Tracking for Assessing Postural Instability in Cerebral Hypoperfusion During Quiet Standing
Postural instability is prevalent in aging and neurodegenerative disease, decreasing quality of life and independence. Quantitatively monitoring balance control is important for assessing treatment efficacy and rehabilit…