Syntactically Look-Ahead Attention Network for Sentence Compression
Sentence compression is the task of compressing a long sentence into a short one by deleting redundant words. In sequence-to-sequence (Seq2Seq) based models, the decoder unidirectionally decides to retain or delete words. Thus, it cannot usually explicitly capture the relationships between decoded words and unseen words that will be decoded in the future time steps. Therefore, to avoid generating ungrammatical sentences, the decoder sometimes drops important words in compressing sentences. To solve this problem, we propose a novel Seq2Seq model, syntactically look-ahead attention network (SLAHAN), that can generate informative summaries by explicitly tracking both dependency parent and child words during decoding and capturing important words that will be decoded in the future. The results of the automatic evaluation on the Google sentence compression dataset showed that SLAHAN achieved the best kept-token-based-F1, ROUGE-1, ROUGE-2 and ROUGE-L scores of 85.5, 79.3, 71.3 and 79.1, respectively. SLAHAN also improved the summarization performance on longer sentences. Furthermore, in the human evaluation, SLAHAN improved informativeness without losing readability.
Code (1)
Tasks
DecoderInformativenessSentenceSentence CompressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Look-ahead Attention for Generation in Neural Machine Translation
The attention model has become a standard component in neural machine translation (NMT) and it guides translation process by selectively focusing on parts of the source sentence when predicting each target word. However,…
Machine TranslationNMTSentenceTranslationStreaming Transformer Transducer Based Speech Recognition Using Non-Causal Convolution
This paper improves the streaming transformer transducer for speech recognition by using non-causal convolution. Many works apply the causal convolution to improve streaming transformer ignoring the lookahead context. We…
speech-recognitionSpeech RecognitionLookahead optimizer improves the performance of Convolutional Autoencoders for reconstruction of natural images
Autoencoders are a class of artificial neural networks which have gained a lot of attention in the recent past. Using the encoder block of an autoencoder the input image can be compressed into a meaningful representation…
Data CompressionDecoderDenoisingLookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammars
Diffusion Large Language Models (dLLMs) have demonstrated promising generative capabilities and are increasingly used to produce formal languages defined by context-free grammars, such as source code and chemical express…
Deleter: Leveraging BERT to Perform Unsupervised Successive Text Compression
Text compression has diverse applications such as Summarization, Reading Comprehension and Text Editing. However, almost all existing approaches require either hand-crafted features, syntactic labels or parallel data. Ev…
Language ModellingReading ComprehensionSentenceSentence Compression+1