Towards Self-Assembling Artificial Neural Networks through Neural Developmental Programs
Biological nervous systems are created in a fundamentally different way than current artificial neural networks. Despite its impressive results in a variety of different domains, deep learning often requires considerable engineering effort to design high-performing neural architectures. By contrast, biological nervous systems are grown through a dynamic self-organizing process. In this paper, we take initial steps toward neural networks that grow through a developmental process that mirrors key properties of embryonic development in biological organisms. The growth process is guided by another neural network, which we call a Neural Developmental Program (NDP) and which operates through local communication alone. We investigate the role of neural growth on different machine learning benchmarks and different optimization methods (evolutionary training, online RL, offline RL, and supervised learning). Additionally, we highlight future research directions and opportunities enabled by having self-organization driving the growth of neural networks.
Code (1)
Tasks
Offline RLSimilar Papers 제목 키워드 기반
Evolving Self-Assembling Neural Networks: From Spontaneous Activity to Experience-Dependent Learning
Biological neural networks are characterized by their high degree of plasticity, a core property that enables the remarkable adaptability of natural organisms. Importantly, this ability affects both the synaptic strength…
Evolved Developmental Artificial Neural Networks for Multitasking with Advanced Activity Dependence
Recently, Cartesian Genetic Programming has been used to evolve developmental programs to guide the formation of artificial neural networks (ANNs). This approach has demonstrated success in enabling ANNs to perform multi…
A topological look into the evolution of developmental programs
Rapid advance of experimental techniques provides an unprecedented in-depth view into complex developmental processes. Still, little is known on how the complexity of multicellular organisms evolved by elaborating develo…
HyperNCA: Growing Developmental Networks with Neural Cellular Automata
In contrast to deep reinforcement learning agents, biological neural networks are grown through a self-organized developmental process. Here we propose a new hypernetwork approach to grow artificial neural networks based…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Growing Artificial Neural Networks for Control: the Role of Neuronal Diversity
In biological evolution complex neural structures grow from a handful of cellular ingredients. As genomes in nature are bounded in size, this complexity is achieved by a growth process where cells communicate locally to …
Diversity