What Have We Achieved on Text Summarization?
Deep learning has led to significant improvement in text summarization with various methods investigated and improved ROUGE scores reported over the years. However, gaps still exist between summaries produced by automatic summarizers and human professionals. Aiming to gain more understanding of summarization systems with respect to their strengths and limits on a fine-grained syntactic and semantic level, we consult the Multidimensional Quality Metric(MQM) and quantify 8 major sources of errors on 10 representative summarization models manually. Primarily, we find that 1) under similar settings, extractive summarizers are in general better than their abstractive counterparts thanks to strength in faithfulness and factual-consistency; 2) milestone techniques such as copy, coverage and hybrid extractive/abstractive methods do bring specific improvements but also demonstrate limitations; 3) pre-training techniques, and in particular sequence-to-sequence pre-training, are highly effective for improving text summarization, with BART giving the best results.
Code (1)
Tasks
Text SummarizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Searching for Effective Neural Extractive Summarization: What Works and What's Next
The recent years have seen remarkable success in the use of deep neural networks on text summarization. However, there is no clear understanding of \textit{why} they perform so well, or \textit{how} they might be improve…
Extractive SummarizationExtractive Text SummarizationText SummarizationAutomatic text summarization: What has been done and what has to be done
Summaries are important when it comes to process huge amounts of information. Their most important benefit is saving time, which we do not have much nowadays. Therefore, a summary must be short, representative and readab…
Text SummarizationAutomatic Summarization of Online Debates
Debate summarization is one of the novel and challenging research areas in automatic text summarization which has been largely unexplored. In this paper, we develop a debate summarization pipeline to summarize key topics…
ClusteringText SummarizationDoes SpatioTemporal information benefit Two video summarization benchmarks?
An important aspect of summarizing videos is understanding the temporal context behind each part of the video to grasp what is and is not important. Video summarization models have in recent years modeled spatio-temporal…
Activity RecognitionVideo SummarizationMachine Learning of Generic and User-Focused Summarization
A key problem in text summarization is finding a salience function which determines what information in the source should be included in the summary. This paper describes the use of machine learning on a training corpus …
BIG-bench Machine LearningText Summarization