Emergence of order in random languages
We consider languages generated by weighted context-free grammars. It is shown that the behaviour of large texts is controlled by saddle-point equations for an appropriate generating function. We then consider ensembles of grammars, in particular the Random Language Model of E. DeGiuli, Phys. Rev. Lett., 122, 128301, 2019. This model is solved in the replica-symmetric ansatz, which is valid in the high-temperature, disordered phase. It is shown that in the phase in which languages carry information, the replica symmetry must be broken.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingvalidSimilar Papers 제목 키워드 기반
NeLLCom-X: A Comprehensive Neural-Agent Framework to Simulate Language Learning and Group Communication
Recent advances in computational linguistics include simulating the emergence of human-like languages with interacting neural network agents, starting from sets of random symbols. The recently introduced NeLLCom framewor…
Random Language Model
Many complex generative systems use languages to create structured objects. We consider a model of random languages, defined by weighted context-free grammars. As the distribution of grammar weights broadens, a transitio…
Language ModelingLanguage ModellingmodelWhen is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfer
While recent work on multilingual language models has demonstrated their capacity for cross-lingual zero-shot transfer on downstream tasks, there is a lack of consensus in the community as to what shared properties betwe…
Cross-Lingual TransferWhen is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfer
While recent work on multilingual language models has demonstrated their capacity for cross-lingual zero-shot transfer on downstream tasks, there is a lack of consensus in the community as to what shared properties betwe…
Cross-Lingual TransferThe Emergence of Compositional Languages for Numeric Concepts Through Iterated Learning in Neural Agents
Since first introduced, computer simulation has been an increasingly important tool in evolutionary linguistics. Recently, with the development of deep learning techniques, research in grounded language learning has also…
Grounded language learning