Wiki Entity Summarization Benchmark
Entity summarization aims to compute concise summaries for entities in knowledge graphs. Existing datasets and benchmarks are often limited to a few hundred entities and discard graph structure in source knowledge graphs. This limitation is particularly pronounced when it comes to ground-truth summaries, where there exist only a few labeled summaries for evaluation and training. We propose WikES, a comprehensive benchmark comprising of entities, their summaries, and their connections. Additionally, WikES features a dataset generator to test entity summarization algorithms in different areas of the knowledge graph. Importantly, our approach combines graph algorithms and NLP models as well as different data sources such that WikES does not require human annotation, rendering the approach cost-effective and generalizable to multiple domains. Finally, WikES is scalable and capable of capturing the complexities of knowledge graphs in terms of topology and semantics. WikES features existing datasets for comparison. Empirical studies of entity summarization methods confirm the usefulness of our benchmark. Data, code, and models are available at: https://github.com/msorkhpar/wiki-entity-summarization.
Code (1)
Tasks
Knowledge GraphsSimilar Papers 제목 키워드 기반
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Abstractive summarization systems leveraging pre-training language models have achieved superior results on benchmark datasets. However, such models have been shown to be more prone to hallucinate facts that are unfaithf…
Abstractive Text SummarizationImproving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Abstractive summarization systems leveraging pre-training language models have achieved superior results on benchmark datasets. However, such models have been shown to be more prone to hallucinate facts that are unfaithf…
Abstractive Text SummarizationTransforming Wikipedia into Augmented Data for Query-Focused Summarization
The limited size of existing query-focused summarization datasets renders training data-driven summarization models challenging. Meanwhile, the manual construction of a query-focused summarization corpus is costly and ti…
Data AugmentationQuery-focused SummarizationGUMsley: Evaluating Entity Salience in Summarization for 12 English Genres
As NLP models become increasingly capable of understanding documents in terms of coherent entities rather than strings, obtaining the most salient entities for each document is not only an important end task in itself bu…
Abstractive Text Summarizationcoreference-resolutionCoreference ResolutionHallucination+2XWikiGen: Cross-lingual Summarization for Encyclopedic Text Generation in Low Resource Languages
Lack of encyclopedic text contributors, especially on Wikipedia, makes automated text generation for low resource (LR) languages a critical problem. Existing work on Wikipedia text generation has focused on English only …
ArticlesCross-Lingual Abstractive SummarizationDocument SummarizationExtractive Summarization+3