In-Context Learning Demonstration Selection via Influence Analysis
Large Language Models (LLMs) have showcased their In-Context Learning (ICL) capabilities, enabling few-shot learning without the need for gradient updates. Despite its advantages, the effectiveness of ICL heavily depends on the choice of demonstrations. Selecting the most effective demonstrations for ICL remains a significant research challenge. To tackle this issue, we propose a demonstration selection method named InfICL, which utilizes influence functions to analyze impacts of training samples. By identifying the most influential training samples as demonstrations, InfICL aims to enhance the ICL generalization performance. To keep InfICL cost-effective, we only use the LLM to generate sample input embeddings, avoiding expensive fine-tuning. Through empirical studies on various real-world datasets, we demonstrate advantages of InfICL compared to state-of-the-art baselines.
Code (1)
Tasks
Few-Shot LearningIn-Context LearningSimilar Papers 제목 키워드 기반
DemoShapley: Valuation of Demonstrations for In-Context Learning
Large language models (LLMs) leveraging in-context learning (ICL) have set new benchmarks in few-shot learning across various tasks without needing task-specific fine-tuning. However, extensive research has demonstrated …
FairnessFew-Shot LearningIn-Context LearningUnraveling Indirect In-Context Learning Using Influence Functions
This work introduces a novel paradigm for generalized In-Context Learning (ICL), termed Indirect In-Context Learning. In Indirect ICL, we explore demonstration selection strategies tailored for two distinct real-world sc…
In-Context LearningInformativenessMMLUStrategyQALearn to Select: Exploring Label Distribution Divergence for In-Context Demonstration Selection in Text Classification
In-context learning (ICL) for text classification, which uses a few input-label demonstrations to describe a task, has demonstrated impressive performance on large language models (LLMs). However, the selection of in-con…
Semantic SimilarityText Classification"Whose Side Are You On?" Estimating Ideology of Political and News Content Using Large Language Models and Few-shot Demonstration Selection
The rapid growth of social media platforms has led to concerns about radicalization, filter bubbles, and content bias. Existing approaches to classifying ideology are limited in that they require extensive human effort, …
ArticlesIn-Context LearningFactors affecting the in-context learning abilities of LLMs for dialogue state tracking
This study explores the application of in-context learning (ICL) to the dialogue state tracking (DST) problem and investigates the factors that influence its effectiveness. We use a sentence embedding based k-nearest nei…
Dialogue State TrackingIn-Context LearningSentenceSentence Embedding+1