paper-with-me

홈 › Papers

Evaluating the Use of Large Language Models as Synthetic Social Agents in Social Science Research

2025-09-30 · Emma Rose Madden arxiv

Large Language Models (LLMs) are being increasingly used as synthetic agents in social science, in applications ranging from augmenting survey responses to powering multi-agent simulations. This paper outlines cautions that should be taken when interpreting LLM outputs and proposes a pragmatic reframing for the social sciences in which LLMs are used as high-capacity pattern matchers for quasi-predictive interpolation under explicit scope conditions and not as substitutes for probabilistic inference. Practical guardrails such as independent draws, preregistered human baselines, reliability-aware validation, and subgroup calibration, are introduced so that researchers may engage in useful prototyping and forecasting while avoiding category errors.

📄 PDF Abstract BibTeX arXiv:2509.26080

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios

2024-12-05 · Xiachong Feng, Longxu Dou, Ella Li, Qinghao Wang 외

Game-theoretic scenarios have become pivotal in evaluating the social intelligence of Large Language Model (LLM)-based social agents. While numerous studies have explored these agents in such settings, there is a lack of…

Language ModelingLanguage ModellingLarge Language ModelSurvey

PersonaArena: Dynamic Simulation for Evaluating and Enhancing Persona-Level Role-Playing in Large Language Models

2026-05-16 · Wenlong Shi, Jianxun Lian, Mingqi Wu, Haiming Qin 외 arxiv

Large language models (LLMs) increasingly serve as interactive social agents, yet their ability to maintain coherent and authentic persona-level role-playing remains limited, particularly in realistic social scenarios. E…

AMONGAGENTS: Evaluating Large Language Models in the Interactive Text-Based Social Deduction Game

2024-07-23 · Yizhou Chi, Lingjun Mao, Zineng Tang

Strategic social deduction games serve as valuable testbeds for evaluating the understanding and inference skills of language models, offering crucial insights into social science, artificial intelligence, and strategic …

Language ModelingLanguage Modelling

AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios

2024-10-25 · Xinyi Mou, Jingcong Liang, Jiayu Lin, Xinnong Zhang 외

Large language models (LLMs) are increasingly leveraged to empower autonomous agents to simulate human beings in various fields of behavioral research. However, evaluating their capacity to navigate complex social intera…

BenchmarkingDiversityNavigate

Evaluating Cultural and Social Awareness of LLM Web Agents

2024-10-30 · Haoyi Qiu, Alexander R. Fabbri, Divyansh Agarwal, Kung-Hsiang Huang 외

As large language models (LLMs) expand into performing as agents for real-world applications beyond traditional NLP tasks, evaluating their robustness becomes increasingly important. However, existing benchmarks often ov…

BenchmarkingNavigate