Bidirectional Context-Aware Hierarchical Attention Network for Document Understanding
The Hierarchical Attention Network (HAN) has made great strides, but it suffers a major limitation: at level 1, each sentence is encoded in complete isolation. In this work, we propose and compare several modifications of HAN in which the sentence encoder is able to make context-aware attentional decisions (CAHAN). Furthermore, we propose a bidirectional document encoder that processes the document forwards and backwards, using the preceding and following sentences as context. Experiments on three large-scale sentiment and topic classification datasets show that the bidirectional version of CAHAN outperforms HAN everywhere, with only a modest increase in computation time. While results are promising, we expect the superiority of CAHAN to be even more evident on tasks requiring a deeper understanding of the input documents, such as abstractive summarization. Code is publicly available.
Code (1)
Tasks
Abstractive Text Summarizationdocument understandingSentenceTopic ClassificationSimilar Papers 제목 키워드 기반
Bidirectional Hierarchical Attention Networks based on Document-level Context for Emotion Cause Extraction
Emotion cause extraction (ECE) aims to extract the causes behind the certain emotion in text. Some works related to the ECE task have been published and attracted lots of attention in recent years. However, these methods…
Emotion Cause ExtractionGraph AttentionGraph-tree Fusion Model with Bidirectional Information Propagation for Long Document Classification
Long document classification presents challenges in capturing both local and global dependencies due to their extensive content and complex structure. Existing methods often struggle with token limits and fail to adequat…
Document ClassificationGraph AttentionSentenceSelective Attention for Context-aware Neural Machine Translation
Despite the progress made in sentence-level NMT, current systems still fall short at achieving fluent, good quality translation for a full document. Recent works in context-aware NMT consider only a few previous sentence…
DecoderMachine TranslationNMTSentence+1Document-Level Neural Machine Translation with Hierarchical Attention Networks
Neural Machine Translation (NMT) can be improved by including document-level contextual information. For this purpose, we propose a hierarchical attention model to capture the context in a structured and dynamic manner. …
DecoderMachine TranslationNMTTranslationDocument Embedding Enhanced Event Detection with Hierarchical and Supervised Attention
Document-level information is very important for event detection even at sentence level. In this paper, we propose a novel Document Embedding Enhanced Bi-RNN model, called DEEB-RNN, to detect events in sentences. This mo…
Document EmbeddingEvent DetectionSentence