paper-with-me

홈 › Papers

A Cross-Linguistic Pressure for Uniform Information Density in Word Order

2023-06-06 · Thomas Hikaru Clark, Clara Meister, Tiago Pimentel, Michael Hahn, Ryan Cotterell, Richard Futrell, Roger Levy

While natural languages differ widely in both canonical word order and word order flexibility, their word orders still follow shared cross-linguistic statistical patterns, often attributed to functional pressures. In the effort to identify these pressures, prior work has compared real and counterfactual word orders. Yet one functional pressure has been overlooked in such investigations: the uniform information density (UID) hypothesis, which holds that information should be spread evenly throughout an utterance. Here, we ask whether a pressure for UID may have influenced word order patterns cross-linguistically. To this end, we use computational models to test whether real orders lead to greater information uniformity than counterfactual orders. In our empirical study of 10 typologically diverse languages, we find that: (i) among SVO languages, real word orders consistently have greater uniformity than reverse word orders, and (ii) only linguistically implausible counterfactual orders consistently exceed the uniformity of real orders. These findings are compatible with a pressure for information uniformity in the development and usage of natural languages.

📄 PDF Abstract BibTeX arXiv:2306.03734

Code (1)

thomashikaru/word-order-uid 공식 구현 pytorch

Tasks

counterfactual

Methods 이 논문이 사용한 방법론

Test 설명 없음

Similar Papers 제목 키워드 기반

Revisiting the Uniform Information Density Hypothesis

2021-09-23 · EMNLP 2021 11 · Clara Meister, Tiago Pimentel, Patrick Haller, Lena Jäger 외

The uniform information density (UID) hypothesis posits a preference among language users for utterances structured such that information is distributed uniformly across a signal. While its implications on language produ…

Linguistic AcceptabilitySentence

Surprise! Uniform Information Density Isn't the Whole Story: Predicting Surprisal Contours in Long-form Discourse

2024-10-21 · Eleftheria Tsipidi, Franz Nowak, Ryan Cotterell, Ethan Wilcox 외

The Uniform Information Density (UID) hypothesis posits that speakers tend to distribute information evenly across linguistic units to achieve efficient communication. Of course, information rate in texts and discourses …

Form

Revisiting Entropy Rate Constancy in Text

2023-05-20 · Vivek Verma, Nicholas Tomlin, Dan Klein

The uniform information density (UID) hypothesis states that humans tend to distribute information roughly evenly across an utterance or discourse. Early evidence in support of the UID hypothesis came from Genzel & Charn…

Is Information Density Uniform when Utterances are Grounded on Perception and Discourse?

2026-02-16 · Matteo Gay, Coleman Haley, Mario Giulianelli, Edoardo Ponti arxiv

The Uniform Information Density (UID) hypothesis posits that speakers are subject to a communicative pressure to distribute information evenly within utterances, minimising surprisal variance. While this hypothesis has b…

Visual Storytelling

Revisiting the UID Hypothesis in LLM Reasoning Traces

2025-10-11 · Minju Gwak, Guijin Son, Jaehyung Kim arxiv

Large language models (LLMs) often solve problems using step-by-step Chain-of-Thought (CoT) reasoning, yet these intermediate steps are frequently unfaithful or hard to interpret. Inspired by the Uniform Information Dens…