Comparing PCG metrics with Human Evaluation in Minecraft Settlement Generation
There are a range of metrics that can be applied to the artifacts produced by procedural content generation, and several of them come with qualitative claims. In this paper, we adapt a range of existing PCG metrics to generated Minecraft settlements, develop a few new metrics inspired by PCG literature, and compare the resulting measurements to existing human evaluations. The aim is to analyze how those metrics capture human evaluation scores in different categories, how the metrics generalize to another game domain, and how metrics deal with more complex artifacts. We provide an exploratory look at a variety of metrics and provide an information gain and several correlation analyses. We found some relationships between human scores and metrics counting specific elements, measuring the diversity of blocks and measuring the presence of crafting materials for the present complex blocks.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityMinecraftSimilar Papers 제목 키워드 기반
Exploring Minecraft Settlement Generators with Generative Shift Analysis
With growing interest in Procedural Content Generation (PCG) it becomes increasingly important to develop methods and tools for evaluating and comparing alternative systems. There is a particular lack regarding the evalu…
MinecraftAutomated Isovist Computation for Minecraft
Procedural content generation for games is a growing trend in both research and industry, even though there is no consensus of how good content looks, nor how to automatically evaluate it. A number of metrics have been d…
MinecraftGenerative Design in Minecraft (GDMC), Settlement Generation Competition
This paper introduces the settlement generation competition for Minecraft, the first part of the Generative Design in Minecraft challenge. The settlement generation competition is about creating Artificial Intelligence (…
MinecraftThe AI Settlement Generation Challenge in Minecraft: First Year Report
This article outlines what we learned from the first year of the AI Settlement Generation Competition in Minecraft, a competition about producing AI programs that can generate interesting settlements in Minecraft for an …
MinecraftGenerative Design in Minecraft: Chronicle Challenge
We introduce the Chronicle Challenge as an optional addition to the Settlement Generation Challenge in Minecraft. One of the foci of the overall competition is adaptive procedural content generation (PCG), an arguably un…
Minecraft