Effective In-Context Example Selection through Data Compression
In-context learning has been extensively validated in large language models. However, the mechanism and selection strategy for in-context example selection, which is a crucial ingredient in this approach, lacks systematic and in-depth research. In this paper, we propose a data compression approach to the selection of in-context examples. We introduce a two-stage method that can effectively choose relevant examples and retain sufficient information about the training dataset within the in-context examples. Our method shows a significant improvement of an average of 5.90% across five different real-world datasets using four language models.
Code (0)
등록된 구현이 없습니다.
Tasks
Data CompressionIn-Context LearningSimilar Papers 제목 키워드 기반
Revisiting In-Context Learning with Long Context Language Models
In-Context Learning (ICL) is a technique by which language models make predictions based on examples provided in their input context. Previously, their context window size imposed a limit on the number of examples that c…
Data AugmentationIn-Context LearningHow Does the Textual Information Affect the Retrieval of Multimodal In-Context Learning?
The increase in parameter size of multimodal large language models (MLLMs) introduces significant capabilities, particularly in-context learning, where MLLMs enhance task performance without updating pre-trained paramete…
In-Context LearningRetrievalThe Role of Diversity in In-Context Learning for Large Language Models
In-context learning (ICL) is a crucial capability of current large language models (LLMs), where the selection of examples plays a key role in performance. While most existing approaches focus on selecting the most simil…
DiversityIn-Context LearningMathSentiment Analysis+1DICE: Dynamic In-Context Example Selection in LLM Agents via Efficient Knowledge Transfer
Large language model-based agents, empowered by in-context learning (ICL), have demonstrated strong capabilities in complex reasoning and tool-use tasks. However, existing works have shown that the effectiveness of ICL i…
Bayesian Example Selection Improves In-Context Learning for Speech, Text, and Visual Modalities
Large language models (LLMs) can adapt to new tasks through in-context learning (ICL) based on a few examples presented in dialogue history without any model parameter update. Despite such convenience, the performance of…
In-Context Learning