SciBERTSUM: Extractive Summarization for Scientific Documents
The summarization literature focuses on the summarization of news articles. The news articles in the CNN-DailyMail are relatively short documents with about 30 sentences per document on average. We introduce SciBERTSUM, our summarization framework designed for the summarization of long documents like scientific papers with more than 500 sentences. SciBERTSUM extends BERTSUM to long documents by 1) adding a section embedding layer to include section information in the sentence vector and 2) applying a sparse attention mechanism where each sentences will attend locally to nearby sentences and only a small number of sentences attend globally to all other sentences. We used slides generated by the authors of scientific papers as reference summaries since they contain the technical details from the paper. The results show the superiority of our model in terms of ROUGE scores.
Code (1)
Tasks
ArticlesExtractive SummarizationSentenceSimilar Papers 제목 키워드 기반
The Effect of Pretraining on Extractive Summarization for Scientific Documents
Large pretrained models have seen enormous success in extractive summarization tasks. In this work, we investigate the influence of pretraining on a BERT-based extractive summarization system for scientific documents. We…
Extractive SummarizationWord EmbeddingsCombination of abstractive and extractive approaches for summarization of long scientific texts
In this research work, we present a method to generate summaries of long scientific documents that uses the advantages of both extractive and abstractive approaches. Before producing a summary in an abstractive manner, w…
Extractive Summarization of Long Documents by Combining Global and Local Context
In this paper, we propose a novel neural single document extractive summarization model for long documents, incorporating both the global context of the whole document and the local context within the current topic. We e…
Extractive SummarizationText SummarizationDocument Summarization with Text Segmentation
In this paper, we exploit the innate document segment structure for improving the extractive summarization task. We build two text segmentation models and find the most optimal strategy to introduce their output predicti…
ArticlesDocument SummarizationExtractive SummarizationSegmentation+2GoSum: Extractive Summarization of Long Documents by Reinforcement Learning and Graph Organized discourse state
Extracting summaries from long documents can be regarded as sentence classification using the structural information of the documents. How to use such structural information to summarize a document is challenging. In thi…
ArticlesDocument SummarizationExtractive Document SummarizationExtractive Summarization+6