Human Evaluation of Creative NLG Systems: An Interdisciplinary Survey on Recent Papers
We survey human evaluation in papers presenting work on creative natural language generation that have been published in INLG 2020 and ICCC 2020. The most typical human evaluation method is a scaled survey, typically on a 5 point scale, while many other less common methods exist. The most commonly evaluated parameters are meaning, syntactic correctness, novelty, relevance and emotional value, among many others. Our guidelines for future evaluation include clearly defining the goal of the generative system, asking questions as concrete as possible, testing the evaluation setup, using multiple different evaluation setups, reporting the entire evaluation process and potential biases clearly, and finally analyzing the evaluation results in a more profound way than merely reporting the most typical statistics.
Code (0)
등록된 구현이 없습니다.
Tasks
SurveyText GenerationSimilar Papers 제목 키워드 기반
Creativity in LLM-based Multi-Agent Systems: A Survey
Large language model (LLM)-driven multi-agent systems (MAS) are transforming how humans and AIs collaboratively generate ideas and artifacts. While existing surveys provide comprehensive overviews of MAS infrastructures,…
Image GenerationLanguage ModelingLanguage ModellingLarge Language Model+1Diffusion-Based Visual Art Creation: A Survey and New Perspectives
The integration of generative AI in visual art has revolutionized not only how visual content is created but also how AI interacts with and reflects the underlying domain knowledge. This survey explores the emerging real…
SurveyStruggle Premium : How Human Effort and Imperfection Drive Perceived Value in the Age of AI
As AI enters creative practice, audiences face growing uncertainty in judging authenticity and value. This study examines the Struggle Premium, the added value attributed to perceived human effort, by analyzing how visib…
Deep Sketch-Based 3D Modeling: A Survey
In the past decade, advances in artificial intelligence have revolutionized sketch-based 3D modeling, leading to a new paradigm known as Deep Sketch-Based 3D Modeling (DS-3DM). DS-3DM offers data-driven methods that addr…
Sparking Scientific Creativity via LLM-Driven Interdisciplinary Inspiration
Despite interdisciplinary research leading to larger and longer-term impact, most work remains confined to single-domain academic silos. Recent AI-based approaches to scientific discovery show promise for interdisciplina…