Guided scenarios with simulated expert personae: a remarkable strategy to perform cognitive work
Large language models (LLMs) trained on a substantial corpus of human knowledge and literature productively work with a large array of facts from that corpus. Surprisingly, they are also able to re-create the behaviors of personae that are captured within the corpus. By forming teams of simulated personae, supplying contexts that set the stage, and providing gentle prompts, one can move through scenarios that elicit expert behavior to perform meaningful cognitive work. The power of this strategy is demonstrated with two examples, one attacking factuality of LLM responses and the other reproducing a very recently published result in quantum optics.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Aligning Large Language Models with Human Opinions through Persona Selection and Value--Belief--Norm Reasoning
Reasoning and predicting human opinions with large language models (LLMs) is essential yet challenging. Current methods employ role-playing with personae but face two major issues: LLMs are sensitive to even a single irr…
PersonaEval: Are LLM Evaluators Human Enough to Judge Role-Play?
Current role-play studies often rely on unvalidated LLM-as-a-judge paradigms, which may fail to reflect how humans perceive role fidelity. A key prerequisite for human-aligned evaluation is role identification, the abili…
Exploring Augmentation and Cognitive Strategies for AI based Synthetic Personae
Large language models (LLMs) hold potential for innovative HCI research, including the creation of synthetic personae. However, their black-box nature and propensity for hallucinations pose challenges. To address these l…
Data AugmentationPositionConcerns on Bias in Large Language Models when Creating Synthetic Personae
This position paper explores the benefits, drawbacks, and ethical considerations of incorporating synthetic personae in HCI research, particularly focusing on the customization challenges beyond the limitations of curren…
PositionWhose Personae? Synthetic Persona Experiments in LLM Research and Pathways to Transparency
Synthetic personae experiments have become a prominent method in Large Language Model alignment research, yet the representativeness and ecological validity of these personae vary considerably between studies. Through a …