Releasing the CRaQAn (Coreference Resolution in Question-Answering): An open-source dataset and dataset creation methodology using instruction-following models
Instruction-following language models demand robust methodologies for information retrieval to augment instructions for question-answering applications. A primary challenge is the resolution of coreferences in the context of chunking strategies for long documents. The critical barrier to experimentation of handling coreferences is a lack of open source datasets, specifically in question-answering tasks that require coreference resolution. In this work we present our Coreference Resolution in Question-Answering (CRaQAn) dataset, an open-source dataset that caters to the nuanced information retrieval requirements of coreference resolution in question-answering tasks by providing over 250 question-answer pairs containing coreferences. To develop this dataset, we developed a novel approach for creating high-quality datasets using an instruction-following model (GPT-4) and a Recursive Criticism and Improvement Loop.
Code (0)
등록된 구현이 없습니다.
Tasks
Chunkingcoreference-resolutionCoreference ResolutionInformation RetrievalInstruction FollowingQuestion AnsweringRetrievalSimilar Papers 제목 키워드 기반
Bridging Context Gaps: Leveraging Coreference Resolution for Long Contextual Understanding
Large language models (LLMs) have shown remarkable capabilities in natural language processing; however, they still face difficulties when tasked with understanding lengthy contexts and executing effective question answe…
coreference-resolutionCoreference ResolutionQuestion AnsweringTowards Harnessing Memory Networks for Coreference Resolution
Coreference resolution task demands comprehending a discourse, especially for anaphoric mentions which require semantic information for resolving antecedents. We investigate into how memory networks can be helpful for co…
coreference-resolutionCoreference ResolutionQuestion AnsweringRepresentation LearningEvaluating Coreference Resolvers on Community-based Question Answering: From Rule-based to State of the Art
Coreference resolution is a key step in natural language understanding. Developments in coreference resolution are mainly focused on improving the performance on standard datasets annotated for coreference resolution. Ho…
Answer Selectioncoreference-resolutionCoreference ResolutionNatural Language Understanding+1Visual Coreference Resolution in Visual Dialog using Neural Module Networks
Visual dialog entails answering a series of questions grounded in an image, using dialog history as context. In addition to the challenges found in visual question answering (VQA), which can be seen as one-round dialog, …
Common Sense Reasoningcoreference-resolutionCoreference ResolutionVisual Dialog+3Improving LLMs' Learning for Coreference Resolution
Coreference Resolution (CR) is crucial for many NLP tasks, but existing LLMs struggle with hallucination and under-performance. In this paper, we investigate the limitations of existing LLM-based approaches to CR-specifi…
Coreference Resolution