Exploring the Role of Diversity in Example Selection for In-Context Learning
In-Context Learning (ICL) has gained prominence due to its ability to perform tasks without requiring extensive training data and its robustness to noisy labels. A typical ICL workflow involves selecting localized examples relevant to a given input using sparse or dense embedding-based similarity functions. However, relying solely on similarity-based selection may introduce topical biases in the retrieved contexts, potentially leading to suboptimal downstream performance. We posit that reranking the retrieved context to enhance topical diversity can improve downstream task performance. To achieve this, we leverage maximum marginal relevance (MMR) which balances topical similarity with inter-example diversity. Our experimental results demonstrate that diversifying the selected examples leads to consistent improvements in downstream performance across various context sizes and similarity functions. The implementation of our approach is made available at https://github.com/janak11111/Diverse-ICL.
Code (1)
Tasks
DiversityIn-Context LearningRerankingSimilar Papers 제목 키워드 기반
The Role of Diversity in In-Context Learning for Large Language Models
In-context learning (ICL) is a crucial capability of current large language models (LLMs), where the selection of examples plays a key role in performance. While most existing approaches focus on selecting the most simil…
DiversityIn-Context LearningMathSentiment Analysis+1Exploring Example Selection for Few-shot Text-to-SQL Semantic Parsing
We study example selection methods for few-shot text-to-SQL tasks with unseen databases. Annotating natural language questions with corresponding SQL queries is expensive, but we can use abundant unlabeled questions to e…
Few-Shot LearningIn-Context LearningSemantic ParsingText to SQL+1Sub-SA: Strengthen In-context Learning via Submodular Selective Annotation
In-context learning (ICL) leverages in-context examples as prompts for the predictions of Large Language Models (LLMs). These prompts play a crucial role in achieving strong performance. However, the selection of suitabl…
DiversityIn-Context LearningVisual Prompt Selection for In-Context Learning Segmentation
As a fundamental and extensively studied task in computer vision, image segmentation aims to locate and identify different semantic concepts at the pixel level. Recently, inspired by In-Context Learning (ICL), several ge…
DiversityImage SegmentationIn-Context LearningSegmentation+1Deep generative selection models of T and B cell receptor repertoires with soNNia
Subclasses of lymphocytes carry different functional roles to work together to produce an immune response and lasting immunity. Additionally to these functional roles, T and B-cell lymphocytes rely on the diversity of th…
BIG-bench Machine LearningDiversity