Brain development dictates energy constraints on neural architecture search: cross-disciplinary insights on optimization strategies
Present day artificial neural architecture search (NAS) strategies are essentially prediction-error-optimized. That holds true for AI functions in general. From the developmental neuroscience perspective, I present evidence for the central role of metabolically, rather than prediction-error-optimized neural architecture search (NAS). Supporting evidence is drawn from the latest insights into the glial-neural organization of the human brain and the dynamic coordination theory which provides a mathematical foundation for the functional expression of this optimization strategy. This is relevant to devising novel NAS strategies in AI, especially in AGI. Additional implications arise for causal reasoning from deep neural nets. Together, the insights from developmental neuroscience offer a new perspective on NAS and the foundational assumptions in AI modeling.
Code (0)
등록된 구현이 없습니다.
Tasks
Neural Architecture SearchSimilar Papers 제목 키워드 기반
Energy Costs and Neural Complexity Evolution in Changing Environments
The Cognitive Buffer Hypothesis (CBH) posits that larger brains evolved to enhance survival in changing conditions. However, larger brains also carry higher energy demands, imposing additional metabolic burdens. Alongsid…
Reinforcement LearningThe Free Energy Principle drives neuromorphic development
We show how any system with morphological degrees of freedom and locally limited free energy will, under the constraints of the free energy principle, evolve toward a neuromorphic morphology that supports hierarchical co…
Energy Storage Management via Deep Q-Networks
Energy storage devices represent environmentally friendly candidates to cope with volatile renewable energy generation. Motivated by the increase in privately owned storage systems, this paper studies the problem of real…
ManagementReinforcement LearningReinforcement Learning (RL)Less is More: some Computational Principles based on Parcimony, and Limitations of Natural Intelligence
Natural intelligence (NI) consistently achieves more with less. Infants learn language, develop abstract concepts, and acquire sensorimotor skills from sparse data, all within tight neural and energy limits. In contrast,…
Disentanglement with Biological Constraints: A Theory of Functional Cell Types
Neurons in the brain are often finely tuned for specific task variables. Moreover, such disentangled representations are highly sought after in machine learning. Here we mathematically prove that simple biological constr…
Disentanglement