Revisiting Entropy Rate Constancy in Text
The uniform information density (UID) hypothesis states that humans tend to distribute information roughly evenly across an utterance or discourse. Early evidence in support of the UID hypothesis came from Genzel & Charniak (2002), which proposed an entropy rate constancy principle based on the probability of English text under n-gram language models. We re-evaluate the claims of Genzel & Charniak (2002) with neural language models, failing to find clear evidence in support of entropy rate constancy. We conduct a range of experiments across datasets, model sizes, and languages and discuss implications for the uniform information density hypothesis and linguistic theories of efficient communication more broadly.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Computational Constancy Measures of Texts---Yule's K and R\'enyi's Entropy
Revisiting and Optimising a CNN Colour Constancy Method for Multi-Illuminant Estimation
The aim of colour constancy is to discount the effect of the scene illumination from the image colours and restore the colours of the objects as captured under a 'white' illuminant. For the majority of colour constancy m…
Revisiting Gray Pixel for Statistical Illumination Estimation
We present a statistical color constancy method that relies on novel gray pixel detection and mean shift clustering. The method, called Mean Shifted Grey Pixel -- MSGP, is based on the observation: true-gray pixels are a…
ClusteringColor ConstancyGenerative Models for Multi-Illumination Color Constancy
In this paper, the aim is multi-illumination color constancy. However, most of the existing color constancy methods are designed for single light sources. Furthermore, datasets for learning multiple illumination color co…
Color ConstancyData AugmentationTranslationMulti-Domain Learning for Accurate and Few-Shot Color Constancy
Color constancy is an important process in camera pipeline to remove the color bias of captured image caused by scene illumination. Recently, significant improvements in color constancy accuracy have been achieved by usi…
Color Constancy