APEER: Automatic Prompt Engineering Enhances Large Language Model Reranking
Large Language Models (LLMs) have significantly enhanced Information Retrieval (IR) across various modules, such as reranking. Despite impressive performance, current zero-shot relevance ranking with LLMs heavily relies on human prompt engineering. Existing automatic prompt engineering algorithms primarily focus on language modeling and classification tasks, leaving the domain of IR, particularly reranking, underexplored. Directly applying current prompt engineering algorithms to relevance ranking is challenging due to the integration of query and long passage pairs in the input, where the ranking complexity surpasses classification tasks. To reduce human effort and unlock the potential of prompt optimization in reranking, we introduce a novel automatic prompt engineering algorithm named APEER. APEER iteratively generates refined prompts through feedback and preference optimization. Extensive experiments with four LLMs and ten datasets demonstrate the substantial performance improvement of APEER over existing state-of-the-art (SoTA) manual prompts. Furthermore, we find that the prompts generated by APEER exhibit better transferability across diverse tasks and LLMs.
Code (0)
등록된 구현이 없습니다.
Tasks
Information RetrievalLanguage ModelingLanguage ModellingLarge Language ModelPrompt EngineeringRerankingRetrievalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
AutoV: Loss-Oriented Ranking for Visual Prompt Retrieval in LVLMs
Inspired by text prompts in large language models, visual prompts have been explored to enhance the perceptual capabilities of large vision-language models (LVLMs). However, performance tends to saturate under single vis…
NeuroPrompts: An Adaptive Framework to Optimize Prompts for Text-to-Image Generation
Despite impressive recent advances in text-to-image diffusion models, obtaining high-quality images often requires prompt engineering by humans who have developed expertise in using them. In this work, we present NeuroPr…
Image GenerationLanguage ModelingLanguage ModellingPrompt Engineering+2Automatic Engineering of Long Prompts
Large language models (LLMs) have demonstrated remarkable capabilities in solving complex open-domain tasks, guided by comprehensive instructions and demonstrations provided in the form of prompts. However, these prompts…
Prompt EngineeringGRAD-SUM: Leveraging Gradient Summarization for Optimal Prompt Engineering
Prompt engineering for large language models (LLMs) is often a manual time-intensive process that involves generating, evaluating, and refining prompts iteratively to ensure high-quality outputs. While there has been wor…
Prompt EngineeringAutomatic Prompt Engineering with No Task Cues and No Tuning
This paper presents a system for automatic prompt engineering that is much simpler in both design and application and yet as effective as the existing approaches. It requires no tuning and no explicit clues about the tas…
Prompt Engineering