Prompt Selection Matters: Enhancing Text Annotations for Social Sciences with Large Language Models
Large Language Models have recently been applied to text annotation tasks from social sciences, equalling or surpassing the performance of human workers at a fraction of the cost. However, no inquiry has yet been made on the impact of prompt selection on labelling accuracy. In this study, we show that performance greatly varies between prompts, and we apply the method of automatic prompt optimization to systematically craft high quality prompts. We also provide the community with a simple, browser-based implementation of the method at https://prompt-ultra.github.io/ .
Code (0)
등록된 구현이 없습니다.
Tasks
text annotationSimilar Papers 제목 키워드 기반
Prompt Design Matters for Computational Social Science Tasks but in Unpredictable Ways
Manually annotating data for computational social science tasks can be costly, time-consuming, and emotionally draining. While recent work suggests that LLMs can perform such annotation tasks in zero-shot settings, littl…
Model SelectionTemporal Prompting Matters: Rethinking Referring Video Object Segmentation
Referring Video Object Segmentation (RVOS) aims to segment the object referred to by the query sentence in the video. Most existing methods require end-to-end training with dense mask annotations, which could be computat…
Referring Video Object SegmentationPromptRL: Prompt Matters in RL for Flow-Based Image Generation
Flow matching models (FMs) have revolutionized text-to-image (T2I) generation, with reinforcement learning (RL) serving as a critical post-training strategy for alignment with reward objectives. In this research, we show…
Reinforcement LearningImage GenerationImage EditingFocus on What Matters: Enhancing Medical Vision-Language Models with Automatic Attention Alignment Tuning
Medical Large Vision-Language Models (Med-LVLMs) often exhibit suboptimal attention distribution on visual inputs, leading to hallucinated or inaccurate outputs. Existing mitigation methods primarily rely on inference-ti…
Visual Question Answering (VQA)Order Matters: Rethinking Prompt Construction in In-Context Learning
In-context learning (ICL) enables large language models to perform new tasks by conditioning on a sequence of examples. Most prior work reasonably and intuitively assumes that which examples are chosen has a far greater …