EASY-M: Evaluation System for Multilingual Summarizers
Automatic text summarization aims at producing a shorter version of a document (or a document set). Evaluation of summarization quality is a challenging task. Because human evaluations are expensive and evaluators often disagree between themselves, many researchers prefer to evaluate their systems automatically, with help of software tools. Such a tool usually requires a point of reference in the form of one or more human-written summaries for each text in the corpus. Then, a system-generated summary is compared to one or more human-written summaries, according to selected metrics. However, a single metric cannot reflect all quality-related aspects of a summary. In this paper we present the EvAluation SYstem for Multilingual Summarization (EASY-M), which enables the evaluation of system-generated summaries in 17 different languages with several quality measures, based on comparison with their human-generated counterparts. The system also provides comparative results with two built-in baselines. The source code and both online and offline versions of EASY-M is freely available for the NLP community.
Code (0)
등록된 구현이 없습니다.
Tasks
Text SummarizationSimilar Papers 제목 키워드 기반
A Principled Framework for Evaluating Summarizers: Comparing Models of Summary Quality against Human Judgments
We present a new framework for evaluating extractive summarizers, which is based on a principled representation as optimization problem. We prove that every extractive summarizer can be decomposed into an objective funct…
PerSum: Novel Systems for Document Summarization in Persian
In this paper we explore the problem of document summarization in Persian language from two distinct angles. In our first approach, we modify a popular and widely cited Persian document summarization framework to see how…
ArticlesDocument SummarizationFacts That Matter
This work introduces fact salience: The task of generating a machine-readable representation of the most prominent information in a text document as a set of facts. We also present SalIE, the first fact salience system. …
ClusteringDiversityEntity LinkingOpen Information Extraction+3A General Optimization Framework for Multi-Document Summarization Using Genetic Algorithms and Swarm Intelligence
Extracting summaries via integer linear programming and submodularity are popular and successful techniques in extractive multi-document summarization. However, many interesting optimization objectives are neither submod…
Document SummarizationMulti-Document SummarizationTrain Hard, Finetune Easy: Multilingual Denoising for RDF-to-Text Generation
We describe our system for the RDF-to-text generation task of the WebNLG Challenge 2020. We base our approach on the mBART model, which is pre-trained for multilingual denoising. This allows us to use a simple, identical…
DenoisingText Generation