Evaluating the Use of Large Language Models as Synthetic Social Agents in Social Science Research
Large Language Models (LLMs) are being increasingly used as synthetic agents in social science, in applications ranging from augmenting survey responses to powering multi-agent simulations. This paper outlines cautions that should be taken when interpreting LLM outputs and proposes a pragmatic reframing for the social sciences in which LLMs are used as high-capacity pattern matchers for quasi-predictive interpolation under explicit scope conditions and not as substitutes for probabilistic inference. Practical guardrails such as independent draws, preregistered human baselines, reliability-aware validation, and subgroup calibration, are introduced so that researchers may engage in useful prototyping and forecasting while avoiding category errors.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios
Game-theoretic scenarios have become pivotal in evaluating the social intelligence of Large Language Model (LLM)-based social agents. While numerous studies have explored these agents in such settings, there is a lack of…
Language ModelingLanguage ModellingLarge Language ModelSurveyPersonaArena: Dynamic Simulation for Evaluating and Enhancing Persona-Level Role-Playing in Large Language Models
Large language models (LLMs) increasingly serve as interactive social agents, yet their ability to maintain coherent and authentic persona-level role-playing remains limited, particularly in realistic social scenarios. E…
AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Game
Strategic social deduction games serve as valuable testbeds for evaluating the understanding and inference skills of language models, offering crucial insights into social science, artificial intelligence, and strategic …
Language ModelingLanguage ModellingAgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios
Large language models (LLMs) are increasingly leveraged to empower autonomous agents to simulate human beings in various fields of behavioral research. However, evaluating their capacity to navigate complex social intera…
BenchmarkingDiversityNavigateEvaluating Cultural and Social Awareness of LLM Web Agents
As large language models (LLMs) expand into performing as agents for real-world applications beyond traditional NLP tasks, evaluating their robustness becomes increasingly important. However, existing benchmarks often ov…
BenchmarkingNavigate