Can Generative Pre-trained Language Models Serve as Knowledge Bases for Closed-book QA?
Recent work has investigated the interesting question using pre-trained language models (PLMs) as knowledge bases for answering open questions. However, existing work is limited in using small benchmarks with high test-train overlaps. We construct a new dataset of closed-book QA using SQuAD, and investigate the performance of BART. Experiments show that it is challenging for BART to remember training facts in high precision, and also challenging to answer closed-book questions even if relevant knowledge is retained. Some promising directions are found, including decoupling the knowledge memorizing process and the QA finetune process, forcing the model to recall relevant knowledge when question answering.
Code (1)
Tasks
Question AnsweringMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Imagine to Hear: Auditory Knowledge Generation can be an Effective Assistant for Language Models
Language models pretrained on text-only corpora often struggle with tasks that require auditory commonsense knowledge. Previous work addresses this problem by augmenting the language model to retrieve knowledge from exte…
Language ModelingLanguage ModellingGenerative Multi-Modal Knowledge Retrieval with Large Language Models
Knowledge retrieval with multi-modal queries plays a crucial role in supporting knowledge-intensive multi-modal applications. However, existing methods face challenges in terms of their effectiveness and training efficie…
RetrievalLanguage Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries
Pretrained language models have been suggested as a possible alternative or complement to structured knowledge bases. However, this emerging LM-as-KB paradigm has so far only been considered in a very limited setting, wh…
World KnowledgeLLM2KB: Constructing Knowledge Bases using instruction tuned context aware Large Language Models
The advent of Large Language Models (LLM) has revolutionized the field of natural language processing, enabling significant progress in various applications. One key area of interest is the construction of Knowledge Base…
Information RetrievalPassage RetrievalRetrievalTransform-Retrieve-Generate: Natural Language-Centric Outside-Knowledge Visual Question Answering
Outside-knowledge visual question answering (OK-VQA) requires the agent to comprehend the image, make use of relevant knowledge from the entire web, and digest all the information to answer the question. Most previou…
Generative Question AnsweringImage to textPassage RetrievalQuestion Answering+3