Parallel Hierarchical Transformer with Attention Alignment for Abstractive Multi-Document Summarization
In comparison to single-document summarization, abstractive Multi-Document Summarization (MDS) brings challenges on the representation and coverage of its lengthy and linked sources. This study develops a Parallel Hierarchical Transformer (PHT) with attention alignment for MDS. By incorporating word- and paragraph-level multi-head attentions, the hierarchical architecture of PHT allows better processing of dependencies at both token and document levels. To guide the decoding towards a better coverage of the source documents, the attention-alignment mechanism is then introduced to calibrate beam search with predicted optimal attention distributions. Based on the WikiSum data, a comprehensive evaluation is conducted to test improvements on MDS by the proposed architecture. By better handling the inner- and cross-document information, results in both ROUGE and human evaluation suggest that our hierarchical model generates summaries of higher quality relative to other Transformer-based baselines at relatively low computational cost.
Code (0)
등록된 구현이 없습니다.
Tasks
Document SummarizationMulti-Document SummarizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Neural Abstractive Multi-Document Summarization: Hierarchical or Flat Structure?
With regards to WikiSum (CITATION) that empowers applicative explorations of Neural Multi-Document Summarization (MDS) to learn from large scale dataset, this study develops two hierarchical Transformers (HT) that descri…
DecoderDocument SummarizationMulti-Document SummarizationWord EmbeddingsImproving Abstractive Dialogue Summarization with Hierarchical Pretraining and Topic Segment
With the increasing abundance of meeting transcripts, meeting summary has attracted more and more attention from researchers. The unsupervised pre-training method based on transformer structure combined with fine-tuning …
Abstractive Dialogue SummarizationArticlesDecoderText Summarization+1MAST: Multimodal Abstractive Summarization with Trimodal Hierarchical Attention
This paper presents MAST, a new model for Multimodal Abstractive Text Summarization that utilizes information from all three modalities -- text, audio and video -- in a multimodal video. Prior work on multimodal abstract…
Abstractive Text SummarizationMultimodal Abstractive Text SummarizationText SummarizationAbstractive Text-Image Summarization Using Multi-Modal Attentional Hierarchical RNN
Rapid growth of multi-modal documents on the Internet makes multi-modal summarization research necessary. Most previous research summarizes texts or images separately. Recent neural summarization research shows the stren…
DecoderExtractive Text SummarizationText SummarizationPAT: Parallel Attention Transformer for Visual Question Answering in Vietnamese
We present in this paper a novel scheme for multimodal learning named the Parallel Attention mechanism. In addition, to take into account the advantages of grammar and context in Vietnamese, we propose the Hierarchical L…
Question AnsweringVietnamese Visual Question AnsweringVisual Question Answering