Neural Abstractive Multi-Document Summarization: Hierarchical or Flat Structure?
With regards to WikiSum (CITATION) that empowers applicative explorations of Neural Multi-Document Summarization (MDS) to learn from large scale dataset, this study develops two hierarchical Transformers (HT) that describe both the cross-token and cross-document dependencies, at the same time allow extended length of input documents. By incorporating word- and paragraph-level multi-head attentions in the decoder based on the parallel and vertical architectures, the proposed parallel and vertical hierarchical Transformers (PHT &VHT) generate summaries utilizing context-aware word embeddings together with static and dynamics paragraph embeddings, respectively. A comprehensive evaluation is conducted on WikiSum to compare PHT &VHT with established models and to answer the question whether hierarchical structures offer more promising performances than flat structures in the MDS task. The results suggest that our hierarchical models generate summaries of higher quality by better capturing cross-document relationships, and save more memory spaces in comparison to flat-structure models. Moreover, we recommend PHT given its practical value of higher inference speed and greater memory-saving capacity.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderDocument SummarizationMulti-Document SummarizationWord EmbeddingsSimilar Papers 제목 키워드 기반
A BART-based approach with hierarchical strategy for Vietnamese abstractive multi-document summarization
In this technical report, we focus on solving the challenge of Vietnamese multi-document abstractive summarization, introduced in the International Workshop on Vietnamese Language and Speech Processing (VLSP) 2022. We ch…
Multi-Document SummarizationAbstractive Text-Image Summarization Using Multi-Modal Attentional Hierarchical RNN
Rapid growth of multi-modal documents on the Internet makes multi-modal summarization research necessary. Most previous research summarizes texts or images separately. Recent neural summarization research shows the stren…
DecoderExtractive Text SummarizationText SummarizationA Discourse-Aware Attention Model for Abstractive Summarization of Long Documents
Neural abstractive summarization models have led to promising results in summarizing relatively short documents. We propose the first model for abstractive summarization of single, longer-form documents (e.g., research p…
Abstractive Text SummarizationDecoderText SummarizationUnsupervised Extractive SummarizationParallel Hierarchical Transformer with Attention Alignment for Abstractive Multi-Document Summarization
In comparison to single-document summarization, abstractive Multi-Document Summarization (MDS) brings challenges on the representation and coverage of its lengthy and linked sources. This study develops a Parallel Hierar…
Document SummarizationMulti-Document SummarizationDomainSum: A Hierarchical Benchmark for Fine-Grained Domain Shift in Abstractive Text Summarization
Most research on abstractive summarization focuses on single-domain applications, often neglecting how domain shifts between documents affect performance and the generalization ability of summarization models. To address…
Abstractive Text SummarizationDomain GeneralizationText Summarization