Neural Architecture Search with Mixed Bio-inspired Learning Rules
Bio-inspired neural networks are attractive for their adversarial robustness, energy frugality, and closer alignment with cortical physiology, yet they often lag behind back-propagation (BP) based models in accuracy and ability to scale. We show that allowing the use of different bio-inspired learning rules in different layers, discovered automatically by a tailored neural-architecture-search (NAS) procedure, bridges this gap. Starting from standard NAS baselines, we enlarge the search space to include bio-inspired learning rules and use NAS to find the best architecture and learning rule to use in each layer. We show that neural networks that use different bio-inspired learning rules for different layers have better accuracy than those that use a single rule across all the layers. The resulting NN that uses a mix of bio-inspired learning rules sets new records for bio-inspired models: 95.16% on CIFAR-10, 76.48% on CIFAR-100, 43.42% on ImageNet16-120, and 60.51% top-1 on ImageNet. In some regimes, they even surpass comparable BP-based networks while retaining their robustness advantages. Our results suggest that layer-wise diversity in learning rules allows better scalability and accuracy, and motivates further research on mixing multiple bio-inspired learning rules in the same network.
Code (0)
등록된 구현이 없습니다.
Tasks
Neural Architecture SearchAdversarial RobustnessResults from the Paper
| Rank | Task | Dataset | Model | Metrics |
|---|---|---|---|---|
| #3 | Adversarial Robustness | CIFAR-10 | Neural Architecture Search with Mixed Bi | Accuracy: 95.16 |
| #4 | Adversarial Robustness | ImageNet | Neural Architecture Search with Mixed Bi | Accuracy: 76.48 |
| #88 | Neural Architecture Search | CIFAR-10 | Neural Architecture Search with Mixed Bi | Accuracy (% ): 95.16 |
| #27 | Neural Architecture Search | CIFAR-100 | Neural Architecture Search with Mixed Bi | Accuracy (% ): 76.48 |
| #275 | Neural Architecture Search | ImageNet | Neural Architecture Search with Mixed Bi | Accuracy: 60.51 |
Similar Papers 제목 키워드 기반
MorphoNAS: Embryogenic Neural Architecture Search Through Morphogen-Guided Development
While biological neural networks develop from compact genomes using relatively simple rules, modern artificial neural architecture search methods mostly involve explicit and routine manual work. In this paper, we introdu…
Neural Architecture SearchNeuromorphic Architecture Optimization for Task-Specific Dynamic Learning
The ability to learn and adapt in real time is a central feature of biological systems. Neuromorphic architectures demonstrating such versatility can greatly enhance our ability to efficiently process information at the …
Meta-LearningAntenna Array Beamforming Based on a Hybrid Quantum Optimization Framework
This paper proposes a hybrid quantum optimization framework for large-scale antenna-array beamforming with jointly optimized discrete phases and continuous amplitudes. The method combines quantum-inspired search with cla…
An AlphaZero-Inspired Approach to Solving Search Problems
AlphaZero and its extension MuZero are computer programs that use machine-learning techniques to play at a superhuman level in chess, go, and a few other games. They achieved this level of play solely with reinforcement …
DisjunctiveNet: Neural Symbolic Learning via Differentiable Convexified Optimization Layers
Many learning tasks in science and engineering are characterized by sparse datasets, which limits the effectiveness of purely data-driven approaches. At the same time, these problems are often accompanied by rich domain …