Zero-Shot Cross-Lingual Abstractive Sentence Summarization through Teaching Generation and Attention
Abstractive Sentence Summarization (ASSUM) targets at grasping the core idea of the source sentence and presenting it as the summary. It is extensively studied using statistical models or neural models based on the large-scale monolingual source-summary parallel corpus. But there is no cross-lingual parallel corpus, whose source sentence language is different to the summary language, to directly train a cross-lingual ASSUM system. We propose to solve this zero-shot problem by using resource-rich monolingual ASSUM system to teach zero-shot cross-lingual ASSUM system on both summary word generation and attention. This teaching process is along with a back-translation process which simulates source-summary pairs. Experiments on cross-lingual ASSUM task show that our proposed method is significantly better than pipeline baselines and previous works, and greatly enhances the cross-lingual performances closer to the monolingual performances.
Code (1)
Tasks
SentenceSentence SummarizationTranslationSimilar Papers 제목 키워드 기반
LEMONADE: A Large Multilingual Expert-Annotated Abstractive Event Dataset for the Real World
This paper presents LEMONADE, a large-scale conflict event dataset comprising 39,786 events across 20 languages and 171 countries, with extensive coverage of region-specific entities. LEMONADE is based on a partially rea…
document understandingEntity LinkingEvent ExtractionZero-shot Event ExtractionImproving Zero-Shot Cross-Lingual Transfer Learning via Robust Training
Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potential for zero-shot cross-lingual transfer. However, these multilingual encoders do not precisely align words and phrases ac…
Cross-Lingual TransferSentencetext-classificationText Classification+3SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic Auto-Encoders
We propose an end-to-end neural model for zero-shot abstractive text summarization of paragraphs, and introduce a benchmark task, ROCSumm, based on ROCStories, a subset for which we collected human summaries. In this tas…
Abstractive Text SummarizationDenoisingSentenceText SummarizationZero-Shot Translation Quality Estimation with Explicit Cross-Lingual Patterns
This paper describes our submission of the WMT 2020 Shared Task on Sentence Level Direct Assessment, Quality Estimation (QE). In this study, we empirically reveal the \textit{mismatching issue} when directly adopting BER…
SentenceTranslationCross-Lingual Natural Language Generation via Pre-Training
In this work we focus on transferring supervision signals of natural language generation (NLG) tasks between multiple languages. We propose to pretrain the encoder and the decoder of a sequence-to-sequence model under bo…
Abstractive Text SummarizationCross-Lingual TransferDecoderMachine Translation+5