Reflect then Learn: Active Prompting for Information Extraction Guided by Introspective Confusion
Large Language Models (LLMs) show remarkable potential for few-shot information extraction (IE), yet their performance is highly sensitive to the choice of in-context examples. Conventional selection strategies often fail to provide informative guidance, as they overlook a key source of model fallibility: confusion stemming not just from semantic content, but also from the generation of well-structured formats required by IE tasks. To address this, we introduce Active Prompting for Information Extraction (APIE), a novel active prompting framework guided by a principle we term introspective confusion. Our method empowers an LLM to assess its own confusion through a dual-component uncertainty metric that uniquely quantifies both Format Uncertainty (difficulty in generating correct syntax) and Content Uncertainty (inconsistency in extracted semantics). By ranking unlabeled data with this comprehensive score, our framework actively selects the most challenging and informative samples to serve as few-shot exemplars. Extensive experiments on four benchmarks show that our approach consistently outperforms strong baselines, yielding significant improvements in both extraction accuracy and robustness. Our work highlights the critical importance of a fine-grained, dual-level view of model uncertainty when it comes to building effective and reliable structured generation systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Information ExtractionSimilar Papers 제목 키워드 기반
X-Reflect: Cross-Reflection Prompting for Multimodal Recommendation
Large Language Models (LLMs) and Large Multimodal Models (LMMs) have been shown to enhance the effectiveness of enriching item descriptions, thereby improving the accuracy of recommendation systems. However, most existin…
Multimodal RecommendationRecommendation SystemsHierarchical Prompting Assists Large Language Model on Web Navigation
Large language models (LLMs) struggle on processing complicated observations in interactive decision making tasks. To alleviate this issue, we propose a simple hierarchical prompting approach. Diverging from previous pro…
Decision MakingLanguage ModelingLanguage ModellingLarge Language ModelFact-and-Reflection (FaR) Improves Confidence Calibration of Large Language Models
For a LLM to be trustworthy, its confidence level should be well-calibrated with its actual performance. While it is now common sense that LLM performances are greatly impacted by prompts, the confidence calibration in p…
Common Sense ReasoningQuestion AnsweringCroPrompt: Cross-task Interactive Prompting for Zero-shot Spoken Language Understanding
Slot filling and intent detection are two highly correlated tasks in spoken language understanding (SLU). Recent SLU research attempts to explore zero-shot prompting techniques in large language models to alleviate the d…
Intent Detectionslot-fillingSlot FillingSpoken Language UnderstandingNext-Best-View Prediction for Active Stereo Cameras and Highly Reflective Objects
Depth acquisition with the active stereo camera is a challenging task for highly reflective objects. When setup permits, multi-view fusion can provide increased levels of depth completion. However, due to the slow acquis…
Depth CompletionPose Estimation