Demonstration Selection for In-Context Learning via Reinforcement Learning

2024-12-05 · Xubin Wang, Jianfei Wu, Yichen Yuan, Mingzhe Li, Deyu Cai, Weijia Jia

Diversity in demonstration selection is crucial for enhancing model generalization, as it enables a broader coverage of structures and concepts. However, constructing an appropriate set of demonstrations has remained a focal point of research. This paper presents the Relevance-Diversity Enhanced Selection (RDES), an innovative approach that leverages reinforcement learning to optimize the selection of diverse reference demonstrations for text classification tasks using Large Language Models (LLMs), especially in few-shot prompting scenarios. RDES employs a Q-learning framework to dynamically identify demonstrations that maximize both diversity and relevance to the classification objective by calculating a diversity score based on label distribution among selected demonstrations. This method ensures a balanced representation of reference data, leading to improved classification accuracy. Through extensive experiments on four benchmark datasets and involving 12 closed-source and open-source LLMs, we demonstrate that RDES significantly enhances classification accuracy compared to ten established baselines. Furthermore, we investigate the incorporation of Chain-of-Thought (CoT) reasoning in the reasoning process, which further enhances the model's predictive performance. The results underscore the potential of reinforcement learning to facilitate adaptive demonstration selection and deepen the understanding of classification challenges.

📄 PDF Abstract BibTeX arXiv:2412.03966

Code (0)

등록된 구현이 없습니다.

Tasks

ClassificationDiversityIn-Context LearningQ-Learningreinforcement-learning Reinforcement Learning text-classification Text Classification

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Learning to Select Visual In-Context Demonstrations

2026-03-24 · Eugene Lee, Yu-Chi Lin, Jiajie Diao arxiv

Multimodal Large Language Models (MLLMs) adapt to visual tasks via in-context learning (ICL), which relies heavily on demonstration quality. The dominant demonstration selection strategy is unsupervised k-Nearest Neighbo…

Reinforcement Learning

Active Example Selection for In-Context Learning

2022-11-08 · Yiming Zhang, Shi Feng, Chenhao Tan

With a handful of demonstration examples, large-scale language models show strong capability to perform various tasks by in-context learning from these examples, without any fine-tuning. We demonstrate that in-context le…

In-Context Learning

Meta-Sel: Efficient Demonstration Selection for In-Context Learning via Supervised Meta-Learning

2026-02-12 · Xubin Wang, Weijia Jia arxiv

Demonstration selection is a practical bottleneck in in-context learning (ICL): under a tight prompt budget, accuracy can change substantially depending on which few-shot examples are included, yet selection must remain …

Reinforcement Learning Intent Classification Prompt Engineering

Learn to Select: Exploring Label Distribution Divergence for In-Context Demonstration Selection in Text Classification

2025-11-10 · Ye Jiang, Taihang Wang, Youzheng Liu, Yimin Wang 외 arxiv

In-context learning (ICL) for text classification, which uses a few input-label demonstrations to describe a task, has demonstrated impressive performance on large language models (LLMs). However, the selection of in-con…

Semantic Similarity Text Classification

A Framework for Learning from Demonstration with Minimal Human Effort

2023-06-15 · Marc Rigter, Bruno Lacerda, Nick Hawes

We consider robot learning in the context of shared autonomy, where control of the system can switch between a human teleoperator and autonomous control. In this setting we address reinforcement learning, and learning fr…

reinforcement-learning Reinforcement Learning