Evaluating the Bias in LLMs for Surveying Opinion and Decision Making in Healthcare
Generative agents have been increasingly used to simulate human behaviour in silico, driven by large language models (LLMs). These simulacra serve as sandboxes for studying human behaviour without compromising privacy or safety. However, it remains unclear whether such agents can truly represent real individuals. This work compares survey data from the Understanding America Study (UAS) on healthcare decision-making with simulated responses from generative agents. Using demographic-based prompt engineering, we create digital twins of survey respondents and analyse how well different LLMs reproduce real-world behaviours. Our findings show that some LLMs fail to reflect realistic decision-making, such as predicting universal vaccine acceptance. However, Llama 3 captures variations across race and Income more accurately but also introduces biases not present in the UAS data. This study highlights the potential of generative agents for behavioural research while underscoring the risks of bias from both LLMs and prompting strategies.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingPrompt EngineeringSurveyMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Evaluating Gender Bias of LLMs in Making Morality Judgements
Large Language Models (LLMs) have shown remarkable capabilities in a multitude of Natural Language Processing (NLP) tasks. However, these models are still not immune to limitations such as social biases, especially gende…
Decision MakingFin-Bias: Comprehensive Evaluation for LLM Decision-Making under human bias in Finance Domain
Large language models (LLMs) are increasingly deployed in financial contexts, raising critical concerns about reliability, alignment, and susceptibility to adversarial manipulation. While prior finance-related benchmarks…
Decoding the Mind of Large Language Models: A Quantitative Evaluation of Ideology and Biases
The widespread integration of Large Language Models (LLMs) across various sectors has highlighted the need for empirical research to understand their biases, thought patterns, and societal implications to ensure ethical …
Biased AI can Influence Political Decision-Making
As modern large language models (LLMs) become integral to everyday tasks, concerns about their inherent biases and their potential impact on human decision-making have emerged. While bias in models are well-documented, l…
Decision MakingEvaluating Large Language Model Biases in Persona-Steered Generation
The task of persona-steered text generation requires large language models (LLMs) to generate text that reflects the distribution of views that an individual fitting a persona could have. People have multifaceted persona…
Language ModelingLanguage ModellingLarge Language ModelMultiple-choice+1