A General Benchmarking Framework for Text Generation
The RDF-to-text task has recently gained substantial attention due to the continuous growth of RDF knowledge graphs in number and size. Recent studies have focused on systematically comparing RDF-to-text approaches on benchmarking datasets such as WebNLG. Although some evaluation tools have already been proposed for text generation, none of the existing solutions abides by the Findability, Accessibility, Interoperability, and Reusability (FAIR) principles and involves RDF data for the knowledge extraction task. In this paper, we present BENG, a FAIR benchmarking platform for Natural Language Generation (NLG) and Knowledge Extraction systems with focus on RDF data. BENG builds upon the successful benchmarking platform GERBIL, is opensource and is publicly available along with the data it contains.
Code (1)
Tasks
BenchmarkingKnowledge GraphsText GenerationSimilar Papers 제목 키워드 기반
A Multifaceted Benchmarking of Synthetic Electronic Health Record Generation Models
Synthetic health data have the potential to mitigate privacy concerns when sharing data to support biomedical research and the development of innovative healthcare applications. Modern approaches for data generation base…
BenchmarkingSynthetic Data GenerationA Framework and Benchmarking Study for Counterfactual Generating Methods on Tabular Data
Counterfactual explanations are viewed as an effective way to explain machine learning predictions. This interest is reflected by a relatively young literature with already dozens of algorithms aiming to generate such ex…
BenchmarkingcounterfactualUniGen: A Unified Framework for Textual Dataset Generation Using Large Language Models
Large Language Models (LLMs) such as GPT-4 and Llama3 have significantly impacted various fields by enabling high-quality synthetic data generation and reducing dependence on expensive human-generated datasets. Despite t…
AttributeBenchmarkingData AugmentationDataset Generation+3QUENCH: Measuring the gap between Indic and Non-Indic Contextual General Reasoning in LLMs
The rise of large language models (LLMs) has created a need for advanced benchmarking systems beyond traditional setups. To this end, we introduce QUENCH, a novel text-based English Quizzing Benchmark manually curated an…
BenchmarkingCommon Sense ReasoningWorld KnowledgeTexygen: A Benchmarking Platform for Text Generation Models
We introduce Texygen, a benchmarking platform to support research on open-domain text generation models. Texygen has not only implemented a majority of text generation models, but also covered a set of metrics that evalu…
BenchmarkingDiversityText Generation