A decentralized route to the origins of scaling in human language
The Zipf's law establishes that if the words of a (large) text are ordered by decreasing frequency, the frequency versus the rank decreases as a power law with exponent close to $-1$. Previous work has stressed that this pattern arises from a conflict of interests of the participants of communication. The challenge here is to define a computational multi-agent language game, mainly based on a parameter that measures the relative participant's interests. Numerical simulations suggest that at critical values of the parameter a human-like vocabulary, exhibiting scaling properties, seems to appear. The appearance of an intermediate distribution of frequencies at some critical values of the parameter suggests that on a population of artificial agents the emergence of scaling partly arises as a self-organized process only from local interactions between agents.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Phase transitions in a decentralized graph-based approach to human language
Zipf's law establishes a scaling behavior for word-frequencies in large text corpora. The appearance of Zipfian properties in human language has been previously explained as an optimization problem for the interests of s…
Baby Scale: Investigating Models Trained on Individual Children's Language Input
Modern language models (LMs) must be trained on many orders of magnitude more words of training data than human children receive before they begin to produce useful behavior. Assessing the nature and origins of this "dat…
Language AcquisitionExplaining Neural Scaling Laws
The population loss of trained deep neural networks often follows precise power-law scaling relations with either the size of the training dataset or the number of parameters in the network. We propose a theory that expl…
From Assumptions to Actions: Turning LLM Reasoning into Uncertainty-Aware Planning for Embodied Agents
Embodied agents operating in multi-agent, partially observable, and decentralized environments must plan and act despite pervasive uncertainty about hidden objects and collaborators' intentions. Recent advances in applyi…
Forecasting Multiple Observables with SCROLL: Score-Trained Uncertainty for Stochastic Dynamics
Forecasting a stochastic dynamical system rarely means a single number: one wants several observables---future state, threshold event, regime label---each with its own likelihood. Standard multi-task recipes balance per-…