paper-with-me

홈 › Papers

Zipf's law emerges asymptotically during phase transitions in communicative systems

2016-03-10 · Bohdan B. Khomtchouk, Claes Wahlestedt

Zipf's law predicts a power-law relationship between word rank and frequency in language communication systems, and is widely reported in texts yet remains enigmatic as to its origins. Computer simulations have shown that language communication systems emerge at an abrupt phase transition in the fidelity of mappings between symbols and objects. Since the phase transition approximates the Heaviside or step function, we show that Zipfian scaling emerges asymptotically at high rank based on the Laplace transform. We thereby demonstrate that Zipf's law gradually emerges from the moment of phase transition in communicative systems. We show that this power-law scaling behavior explains the emergence of natural languages at phase transitions. We find that the emergence of Zipf's law during language communication suggests that the use of rare words in a lexicon is critical for the construction of an effective communicative system at the phase transition.

📄 PDF Abstract BibTeX arXiv:1603.03153

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Phase transitions in a decentralized graph-based approach to human language

2020-03-04 · Javier Vera, Felipe Urbina, Wenceslao Palma

Zipf's law establishes a scaling behavior for word-frequencies in large text corpora. The appearance of Zipfian properties in human language has been previously explained as an optimization problem for the interests of s…

Evidence of Phase Transitions in Small Transformer-Based Language Models

2025-11-16 · Noah Hong, Tao Hong arxiv

Phase transitions have been proposed as the origin of emergent abilities in large language models (LLMs), where new capabilities appear abruptly once models surpass critical thresholds of scale. Prior work, such as that …

Pragmatic Constraint on Distributional Semantics

2022-11-20 · Elizaveta Zhemchuzhina, Nikolai Filippov, Ivan P. Yamshchikov

This paper studies the limits of language models' statistical learning in the context of Zipf's law. First, we demonstrate that Zipf-law token distribution emerges irrespective of the chosen tokenization. Second, we show…

Relating Turing's Formula and Zipf's Law

1996-06-11 · Christer Samuelsson

An asymptote is derived from Turing's local reestimation formula for population frequencies, and a local reestimation formula is derived from Zipf's law for the asymptotic behavior of population frequencies. The two are …

Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition

2023-10-10 · Zhongtian Chen, Edmund Lau, Jake Mendel, Susan Wei 외

We investigate phase transitions in a Toy Model of Superposition (TMS) using Singular Learning Theory (SLT). We derive a closed formula for the theoretical loss and, in the case of two hidden dimensions, discover that re…

Learning Theory