Analysing Data-To-Text Generation Benchmarks
Recently, several data-sets associating data to text have been created to train data-to-text surface realisers. It is unclear however to what extent the surface realisation task exercised by these data-sets is linguistically challenging. Do these data-sets provide enough variety to encourage the development of generic, high-quality data-to-text surface realisers ? In this paper, we argue that these data-sets have important drawbacks. We back up our claim using statistics, metrics and manual evaluation. We conclude by eliciting a set of criteria for the creation of a data-to-text benchmark which could help better support the development, evaluation and comparison of linguistically sophisticated data-to-text surface realisers.
Code (0)
등록된 구현이 없습니다.
Tasks
Data-to-Text GenerationText GenerationSimilar Papers 제목 키워드 기반
Is my model "mind blurting"? Interpreting the dynamics of reasoning tokens with Recurrence Quantification Analysis (RQA)
Test-time compute is central to large reasoning models, yet analysing their reasoning behaviour through generated text is increasingly impractical and unreliable. Response length is often used as a brute proxy for reason…
Reading Text in the Wild with Convolutional Neural Networks
In this work we present an end-to-end system for text spotting -- localising and recognising text in natural scene images -- and text based image retrieval. This system is based on a region proposal mechanism for detecti…
Image RetrievalRegion ProposalRetrievalText Generation+1Post-edit Analysis of Collective Biography Generation
Text generation is increasingly common but often requires manual post-editing where high precision is critical to end users. However, manual editing is expensive so we want to ensure this effort is focused on high-value …
Text GenerationNatural Language Generation as Planning under Uncertainty Using Reinforcement Learning
We present and evaluate a new model for Natural Language Generation (NLG) in Spoken Dialogue Systems, based on statistical planning, given noisy feedback from the current generation context (e.g. a user and a surface rea…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Spoken Dialogue Systems+1TextDescriptives: A Python package for calculating a large variety of metrics from text
TextDescriptives is a Python package for calculating a large variety of metrics from text. It is built on top of spaCy and can be easily integrated into existing workflows. The package has already been used for analysing…