Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits
Brain-inspired machine intelligence research seeks to develop computational models that emulate the information processing and adaptability that distinguishes biological systems of neurons. This has led to the development of spiking neural networks, a class of models that promisingly addresses the biological implausibility and {the lack of energy efficiency} inherent to modern-day deep neural networks. In this work, we address the challenge of designing neurobiologically-motivated schemes for adjusting the synapses of spiking networks and propose contrastive-signal-dependent plasticity, a process which generalizes ideas behind self-supervised learning to facilitate local adaptation in architectures of event-based neuronal layers that operate in parallel. Our experimental simulations demonstrate a consistent advantage over other biologically-plausible approaches when training recurrent spiking networks, crucially side-stepping the need for extra structure such as feedback synapses.
Code (1)
Tasks
Self-Supervised LearningSimilar Papers 제목 키워드 기반
Self-supervised local learning rules learn the hidden hierarchical structure of high-dimensional data
The brain learns abstract representations of high-dimensional sensory input, but the plasticity rules that enable such learning are unknown. We study biologically plausible algorithms on the Random Hierarchy Model (RHM),…
Local plasticity rules can learn deep representations using self-supervised contrastive predictions
Learning in the brain is poorly understood and learning rules that respect biological constraints, yet yield deep hierarchical representations, are still unknown. Here, we propose a learning rule that takes inspiration f…
Learning of Precise Spike Times with Membrane Potential Dependent Synaptic Plasticity
Precise spatio-temporal patterns of neuronal action potentials underly e.g. sensory representations and control of muscle activities. However, it is not known how the synaptic efficacies in the neuronal networks of the b…
RetrievalSensitivityUnsupervised Contrastive Learning for Signal-Dependent Noise Synthesis
We present a simple yet robust noise synthesis framework based on unsupervised contrastive learning. With access to clean images only, the proposed contrastive noise synthesis framework trains a Glow-based generative mod…
Contrastive LearningSelf-Motivated Growing Neural Network for Adaptive Architecture via Local Structural Plasticity
Control policies in deep reinforcement learning are often implemented with fixed-capacity multilayer perceptrons trained by backpropagation, which lack structural plasticity and depend on global error signals. This paper…
Reinforcement Learning