Continual BERT: Continual Learning for Adaptive Extractive Summarization of COVID-19 Literature
The scientific community continues to publish an overwhelming amount of new research related to COVID-19 on a daily basis, leading to much literature without little to no attention. To aid the community in understanding the rapidly flowing array of COVID-19 literature, we propose a novel BERT architecture that provides a brief yet original summarization of lengthy papers. The model continually learns on new data in online fashion while minimizing catastrophic forgetting, thus fitting to the need of the community. Benchmark and manual examination of its performance show that the model provide a sound summary of new scientific literature.
Code (1)
Tasks
Continual LearningExtractive SummarizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DACP: Domain-Adaptive Continual Pre-Training of Large Language Models for Phone Conversation Summarization
Large language models (LLMs) have achieved impressive performance in text summarization, yet their performance often falls short when applied to specialized domains that differ from their original pre-training distributi…
Text SummarizationBERT Fine-tuning For Arabic Text Summarization
Fine-tuning a pretrained BERT model is the state of the art method for extractive/abstractive text summarization, in this paper we showcase how this fine-tuning method can be applied to the Arabic language to both constr…
Abstractive Text SummarizationExtractive SummarizationText SummarizationContinually Improving Extractive QA via Human Feedback
We study continually improving an extractive question answering (QA) system via human user feedback. We design and deploy an iterative approach, where information-seeking users ask questions, receive model-predicted answ…
Domain AdaptationExtractive Question-AnsweringQuestion AnsweringEvaluating Extractive Summarization Techniques on News Articles
In recent years, due to the rise of deep learning and natural language processing, text summarization has become a huge topic among scholars. Text summarization derives a shorter coherent version of a longer document. Th…
ArticlesExtractive SummarizationText SummarizationEvaluating Extractive Summarization Techniques on News Articles
In recent years, due to the rise of deep learning and natural language processing, text summarization has become a huge topic among scholars. Text summarization derives a shorter coherent version of a longer document. Th…
ArticlesExtractive SummarizationText Summarization