paper-with-me

홈 › Papers

'Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participants

2024-09-28 · Shivani Kapania, William Agnew, Motahhare Eslami, Hoda Heidari, Sarah Fox

The recent excitement around generative models has sparked a wave of proposals suggesting the replacement of human participation and labor in research and development--e.g., through surveys, experiments, and interviews--with synthetic research data generated by large language models (LLMs). We conducted interviews with 19 qualitative researchers to understand their perspectives on this paradigm shift. Initially skeptical, researchers were surprised to see similar narratives emerge in the LLM-generated data when using the interview probe. However, over several conversational turns, they went on to identify fundamental limitations, such as how LLMs foreclose participants' consent and agency, produce responses lacking in palpability and contextual depth, and risk delegitimizing qualitative research methods. We argue that the use of LLMs as proxies for participants enacts the surrogate effect, raising ethical and epistemological concerns that extend beyond the technical limitations of current models to the core of whether LLMs fit within qualitative ways of knowing.

📄 PDF Abstract BibTeX arXiv:2409.19430

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents

2024-05-05 · Junkai Li, Yunghwei Lai, Weitao Li, Jingyi Ren 외

The recent rapid development of large language models (LLMs) has sparked a new wave of technological revolution in medical artificial intelligence (AI). While LLMs are designed to understand and generate text like a huma…

MedQAQuestion Answering

Do Language Models' Words Refer?

2023-08-10 · Matthew Mandelkern, Tal Linzen

What do language models (LMs) do with language? Everyone agrees that they can produce sequences of (mostly) coherent strings of English. But do those sentences mean something, or are LMs simply babbling in a convincing s…

Philosophy

Small Language Models can Outperform Humans in Short Creative Writing: A Study Comparing SLMs with Humans and LLMs

2024-09-17 · Guillermo Marco, Luz Rello, Julio Gonzalo

In this paper, we evaluate the creative fiction writing abilities of a fine-tuned small language model (SLM), BART-large, and compare its performance to human writers and two large language models (LLMs): GPT-3.5 and GPT…

Language ModellingSmall Language Model

Clustering Discourses: Racial Biases in Short Stories about Women Generated by Large Language Models

2025-09-02 · Gustavo Bonil, João Gondim, Marina dos Santos, Simone Hashiguti 외 arxiv

This study investigates how large language models, in particular LLaMA 3.2-3B, construct narratives about Black and white women in short stories generated in Portuguese. From 2100 texts, we applied computational methods …

Cut the CARP: Fishing for zero-shot story evaluation

2021-10-06 · Shahbuland Matiana, JR Smith, Ryan Teehan, Louis Castricato 외

Recent advances in large-scale language models (Raffel et al., 2019; Brown et al., 2020) have brought significant qualitative and quantitative improvements in machine-driven text generation. Despite this, generation and …

Contrastive LearningLanguage ModelingLanguage ModellingPrompt Engineering+1