Expressivity of Emergent Languages is a Trade-off between Contextual Complexity and Unpredictability
Researchers are using deep learning models to explore the emergence of language in various language games, where simulated agents interact and develop an emergent language to solve a task. We focus on factors which determine the expressivity of emergent languages, which reflects the amount of information about input spaces those languages are capable of encoding. We measure the expressivity of emergent languages based on their generalisation performance across different games, and demonstrate that the expressivity of emergent languages is a trade-off between the complexity and unpredictability of the context those languages are used in. Another novel contribution of this work is introducing a contrastive loss into the learning methods on referential games. We show that using our contrastive loss alleviates the collapse of message types seen using standard referential loss functions.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Expressivity of Emergent Language is a Trade-off between Contextual Complexity and Unpredictability
Researchers are using deep learning models to explore the emergence of language in various language games, where agents interact and develop an emergent language to solve tasks. We focus on the factors that determine the…
Inductive Bias and Language Expressivity in Emergent Communication
Referential games and reconstruction games are the most common game types for studying emergent languages. We investigate how the type of the language game affects the emergent language in terms of: i) language compositi…
Inductive BiasA Rich Morphological Tagger for English: Exploring the Cross-Linguistic Tradeoff Between Morphology and Syntax
A traditional claim in linguistics is that all human languages are equally expressive{---}able to convey the same wide range of meanings. Morphologically rich languages, such as Czech, rely on overt inflectional and deri…
Dependency ParsingMachine TranslationMorphological AnalysisMorphological Tagging+1The Bias-Expressivity Trade-off
Learning algorithms need bias to generalize and perform better than random guessing. We examine the flexibility (expressivity) of biased algorithms. An expressive algorithm can adapt to changing training data, altering i…
Bridging the Stability-Expressivity Gap: Synthetic Data Scaling and Preference Alignment for Low-Resource Spoken Language Models
Spoken Language Models (SLMs) have emerged as a promising paradigm for speech synthesis by bypassing explicit grapheme-to-phoneme pipelines. However, their effectiveness in low-resource languages remains fundamentally li…
Speech Synthesis