Oja's plasticity rule overcomes several challenges of training neural networks under biological constraints
Deep neural networks have achieved impressive performance through carefully engineered training strategies. Nonetheless, such methods lack parallels in biological neural circuits, relying heavily on non-local credit assignment, precise initialization, normalization layers, batch processing, and large datasets. Biologically plausible plasticity rules, such as random feedback alignment, often suffer from instability and unbounded weight growth without these engineered methods, while Hebbian-type schemes fail to provide goal-oriented credit. In this study, we demonstrate that incorporating Oja's plasticity rule into error-driven training yields stable, efficient learning in feedforward and recurrent architectures, obviating the need for carefully engineered tricks. Our results show that Oja's rule preserves richer activation subspaces, mitigates exploding or vanishing signals, and improves short-term memory in recurrent networks. Notably, meta-learned local plasticity rules incorporating Oja's principle not only match but surpass standard backpropagation in data-scarce regimes. These findings reveal a biologically grounded pathway bridging engineered deep networks and plausible synaptic mechanisms.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Meta-learning families of plasticity rules in recurrent spiking networks using simulation-based inference
There is substantial experimental evidence that learning and memory-related behaviours rely on local synaptic changes, but the search for distinct plasticity rules has been driven by human intuition, with limited success…
A meta-learning approach to (re)discover plasticity rules that carve a desired function into a neural network
The search for biologically faithful synaptic plasticity rules has resulted in a large body of models. They are usually inspired by -- and fitted to -- experimental data, but they rarely produce neural dynamics that serv…
Meta-LearningHebbian and Gradient-based Plasticity Enables Robust Memory and Rapid Learning in RNNs
Rapidly learning from ongoing experiences and remembering past events with a flexible memory system are two core capacities of biological intelligence. While the underlying neural mechanisms are not fully understood, var…
Few-Shot LearningSpikePropamine: Differentiable Plasticity in Spiking Neural Networks
The adaptive changes in synaptic efficacy that occur between spiking neurons have been demonstrated to play a critical role in learning for biological neural networks. Despite this source of inspiration, many learning fo…
Dis-inhibitory neuronal circuits can control the sign of synaptic plasticity
How neuronal circuits achieve credit assignment remains a central unsolved question in systems neuroscience. Various studies have suggested plausible solutions for back-propagating error signals through multi-layer netwo…