Searching for Structure: Investigating Emergent Communication with Large Language Models
Human languages have evolved to be structured through repeated language learning and use. These processes introduce biases that operate during language acquisition and shape linguistic systems toward communicative efficiency. In this paper, we investigate whether the same happens if artificial languages are optimised for implicit biases of Large Language Models (LLMs). To this end, we simulate a classical referential game in which LLMs learn and use artificial languages. Our results show that initially unstructured holistic languages are indeed shaped to have some structural properties that allow two LLM agents to communicate successfully. Similar to observations in human experiments, generational transmission increases the learnability of languages, but can at the same time result in non-humanlike degenerate vocabularies. Taken together, this work extends experimental findings, shows that LLMs can be used as tools in simulations of language evolution, and opens possibilities for future human-machine experiments in this field.
Code (0)
등록된 구현이 없습니다.
Tasks
Language AcquisitionSimilar Papers 제목 키워드 기반
Vehicle Communication Strategies for Simulated Highway Driving
Interest in emergent communication has recently surged in Machine Learning. The focus of this interest has largely been either on investigating the properties of the learned protocol or on utilizing emergent communicatio…
BIG-bench Machine LearningSelf-Driving CarsSearching for the Most Human-like Emergent Language
In this paper, we design a signalling game-based emergent communication environment to generate state-of-the-art emergent languages in terms of similarity to human language. This is done with hyperparameter optimization,…
Hyperparameter OptimizationTransfer LearningEmergent Communication with Attention
To develop computational agents that better communicate using their own emergent language, we endow the agents with an ability to focus their attention on particular concepts in the environment. Humans often understand a…
Neuron-based Multifractal Analysis of Neuron Interaction Dynamics in Large Models
In recent years, there has been increasing attention on the capabilities of large models, particularly in handling complex tasks that small-scale models are unable to perform. Notably, large language models (LLMs) have d…
Categorial Grammar Induction as a Compositionality Measure for Understanding the Structure of Emergent Languages
This paper proposes a method for investigating the syntactic structure of emergent languages using categorial grammar induction. Although the structural property of emergent languages is an important topic, little has be…
Sentence