Contri(e)ve: Context + Retrieve for Scholarly Question Answering
Scholarly communication is a rapid growing field containing a wealth of knowledge. However, due to its unstructured and document format, it is challenging to extract useful information from them through conventional document retrieval methods. Scholarly knowledge graphs solve this problem, by representing the documents in a semantic network, providing, hidden insights, summaries and ease of accessibility through queries. Naturally, question answering for scholarly graphs expands the accessibility to a wider audience. But some of the knowledge in this domain is still presented as unstructured text, thus requiring a hybrid solution for question answering systems. In this paper, we present a two step solution using open source Large Language Model(LLM): Llama3.1 for Scholarly-QALD dataset. Firstly, we extract the context pertaining to the question from different structured and unstructured data sources: DBLP, SemOpenAlex knowledge graphs and Wikipedia text. Secondly, we implement prompt engineering to improve the information retrieval performance of the LLM. Our approach achieved an F1 score of 40% and also observed some anomalous responses from the LLM, that are discussed in the final part of the paper.
Code (0)
등록된 구현이 없습니다.
Tasks
Information RetrievalKnowledge GraphsLanguage ModelingLanguage ModellingLarge Language ModelPrompt EngineeringQuestion AnsweringRetrievalSimilar Papers 제목 키워드 기반
NLQxform: A Language Model-based Question to SPARQL Transformer
In recent years, scholarly data has grown dramatically in terms of both scale and complexity. It becomes increasingly challenging to retrieve information from scholarly knowledge graphs that include large-scale heterogen…
Graph Question AnsweringKnowledge GraphsLanguage ModelingLanguage Modelling+1Leveraging LLMs in Scholarly Knowledge Graph Question Answering
This paper presents a scholarly Knowledge Graph Question Answering (KGQA) that answers bibliographic natural language questions by leveraging a large language model (LLM) in a few-shot manner. The model initially identif…
Graph Question AnsweringLanguage ModelingLanguage ModellingLarge Language Model+2Question Answering on Scholarly Knowledge Graphs
Answering questions on scholarly knowledge comprising text and other artifacts is a vital part of any research life cycle. Querying scholarly knowledge and retrieving suitable answers is currently hardly possible due to …
ArticlesKnowledge Base Question AnsweringKnowledge GraphsQuestion AnsweringIntegrating SPARQL and LLMs for Question Answering over Scholarly Data Sources
The Scholarly Hybrid Question Answering over Linked Data (QALD) Challenge at the International Semantic Web Conference (ISWC) 2024 focuses on Question Answering (QA) over diverse scholarly sources: DBLP, SemOpenAlex, and…
Extractive Question-AnsweringQuestion AnsweringDetect, Retrieve, Comprehend: A Flexible Framework for Zero-Shot Document-Level Question Answering
Researchers produce thousands of scholarly documents containing valuable technical knowledge. The community faces the laborious task of reading these documents to identify, extract, and synthesize information. To automat…
Question AnsweringRetrieval