STAYKATE: Hybrid In-Context Example Selection Combining Representativeness Sampling and Retrieval-based Approach -- A Case Study on Science Domains
Large language models (LLMs) demonstrate the ability to learn in-context, offering a potential solution for scientific information extraction, which often contends with challenges such as insufficient training data and the high cost of annotation processes. Given that the selection of in-context examples can significantly impact performance, it is crucial to design a proper method to sample the efficient ones. In this paper, we propose STAYKATE, a static-dynamic hybrid selection method that combines the principles of representativeness sampling from active learning with the prevalent retrieval-based approach. The results across three domain-specific datasets indicate that STAYKATE outperforms both the traditional supervised methods and existing selection methods. The enhancement in performance is particularly pronounced for entity types that other methods pose challenges.
Code (0)
등록된 구현이 없습니다.
Tasks
Active LearningRetrievalSimilar Papers 제목 키워드 기반
Learning What Helps: Task-Aligned Context Selection for Vision Tasks
Humans often resolve visual uncertainty by comparing an image with relevant examples, but ViTs lack the ability to identify which examples would improve their predictions. We present Task-Aligned Context Selection (TACS)…
Medical Image ClassificationMedical Image SegmentationReinforcement LearningFew-Shot Optimization for Sensor Data Using Large Language Models: A Case Study on Fatigue Detection
In this paper, we propose a novel few-shot optimization with HED-LM (Hybrid Euclidean Distance with Large Language Models) to improve example selection for sensor-based classification tasks. While few-shot prompting enab…
Activity RecognitionHuman Activity RecognitionCTQScorer: Combining Multiple Features for In-context Example Selection for Machine Translation
Large language models have demonstrated the capability to perform on machine translation when the input is prompted with a few examples (in-context learning). Translation quality depends on various features of the select…
In-Context LearningMachine TranslationRetrievalTranslationGoing Beyond Word Matching: Syntax Improves In-context Example Selection for Machine Translation
In-context learning (ICL) is the trending prompting strategy in the era of large language models (LLMs), where a few examples are demonstrated to evoke LLMs' power for a given task. How to select informative examples rem…
In-Context LearningMachine TranslationTranslationHybrid Neural-LLM Pipeline for Morphological Glossing in Endangered Language Documentation: A Case Study of Jungar Tuvan
Interlinear glossed text (IGT) creation remains a major bottleneck in linguistic documentation and fieldwork, particularly for low-resource morphologically rich languages. We present a hybrid automatic glossing pipeline …
Structured Prediction