Which Information Matters? Dissecting Human-written Multi-document Summaries with Partial Information Decomposition
Understanding the nature of high-quality summaries is crucial to further improve the performance of multi-document summarization. We propose an approach to characterize human-written summaries using partial information decomposition, which decomposes the mutual information provided by all source documents into union, redundancy, synergy, and unique information. Our empirical analysis on different MDS datasets shows that there is a direct dependency between the number of sources and their contribution to the summary.
Code (1)
Tasks
Document SummarizationMulti-Document SummarizationSimilar Papers 제목 키워드 기반
Towards Tailored Recovery of Lexical Diversity in Literary Machine Translation
Machine translations are found to be lexically poorer than human translations. The loss of lexical diversity through MT poses an issue in the automatic translation of literature, where it matters not only what is written…
DiversityMachine TranslationRerankingTranslationThe Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts
As the influence of large language models (LLMs) spans across global communities, their safety challenges in multilingual settings become paramount for alignment research. This paper examines the variations in safety cha…
Unveiling Scoring Processes: Dissecting the Differences between LLMs and Human Graders in Automatic Scoring
Large language models (LLMs) have demonstrated strong potential in performing automatic scoring for constructed response assessments. While constructed responses graded by humans are usually based on given grading rubric…
Logical ReasoningMultiSocial: Multilingual Benchmark of Machine-Generated Text Detection of Social-Media Texts
Recent LLMs are able to generate high-quality multilingual texts, indistinguishable for humans from authentic human-written ones. Research in machine-generated text detection is however mostly focused on the English lang…
ArticlesBenchmarkingText DetectionFully Synthetic Data Improves Neural Machine Translation with Knowledge Distillation
This paper explores augmenting monolingual data for knowledge distillation in neural machine translation. Source language monolingual text can be incorporated as a forward translation. Interestingly, we find the best way…
Knowledge DistillationMachine TranslationTranslation