Enhancing In-Context Learning with Answer Feedback for Multi-Span Question Answering
Whereas the recent emergence of large language models (LLMs) like ChatGPT has exhibited impressive general performance, it still has a large gap with fully-supervised models on specific tasks such as multi-span question answering. Previous researches found that in-context learning is an effective approach to exploiting LLM, by using a few task-related labeled data as demonstration examples to construct a few-shot prompt for answering new questions. A popular implementation is to concatenate a few questions and their correct answers through simple templates, informing LLM of the desired output. In this paper, we propose a novel way of employing labeled data such that it also informs LLM of some undesired output, by extending demonstration examples with feedback about answers predicted by an off-the-shelf model, e.g., correct, incorrect, or incomplete. Experiments on three multi-span question answering datasets as well as a keyphrase extraction dataset show that our new prompting strategy consistently improves LLM's in-context learning performance.
Code (3)
Tasks
In-Context LearningKeyphrase ExtractionQuestion AnsweringSimilar Papers 제목 키워드 기반
Localizing and Mitigating Errors in Long-form Question Answering
Long-form question answering (LFQA) aims to provide thorough and in-depth answers to complex questions, enhancing comprehension. However, such detailed responses are prone to hallucinations and factual inconsistencies, c…
FormHallucinationLong Form Question AnsweringQuestion AnsweringCorrect after Answer: Enhancing Multi-Span Question Answering with Post-Processing Method
Multi-Span Question Answering (MSQA) requires models to extract one or multiple answer spans from a given context to answer a question. Prior work mainly focuses on designing specific methods or applying heuristic strate…
Question AnsweringMULTI-SPAN QUESTION ANSWERING USING SPAN-IMAGE NETWORK
Question-answering (QA) models aim to find an answer given a question and con- text. Language models like BERT are used to associate question and context to find an answer span. Prior art on QA focuses on finding the bes…
Question AnsweringvalidPost-Training Large Language Models via Reinforcement Learning from Self-Feedback
Large Language Models (LLMs) often produce plausible but poorly-calibrated answers, limiting their reliability on reasoning-intensive tasks. Recent research suggests that Chain-of-Thought (CoT) reasoning paths are inhere…
Reinforcement LearningArithmetic ReasoningQuestion AnsweringEvaluating Vision-Language and Large Language Models for Automated Student Assessment in Indonesian Classrooms
Although vision-language and large language models (VLM and LLM) offer promising opportunities for AI-driven educational assessment, their effectiveness in real-world classroom settings, particularly in underrepresented …
Multiple-choice