Winner-Take-All as Basic Probabilistic Inference Unit of Neuronal Circuits
Experimental observations of neuroscience suggest that the brain is working a probabilistic way when computing information with uncertainty. This processing could be modeled as Bayesian inference. However, it remains unclear how Bayesian inference could be implemented at the level of neuronal circuits of the brain. In this study, we propose a novel general-purpose neural implementation of probabilistic inference based on a ubiquitous network of cortical microcircuits, termed winner-take-all (WTA) circuit. We show that each WTA circuit could encode the distribution of states defined on a variable. By connecting multiple WTA circuits together, the joint distribution can be represented for arbitrary probabilistic graphical models. Moreover, we prove that the neural dynamics of WTA circuit is able to implement one of the most powerful inference methods in probabilistic graphical models, mean-field inference. We show that the synaptic drive of each spiking neuron in the WTA circuit encodes the marginal probability of the variable in each state, and the firing probability (or firing rate) of each neuron is proportional to the marginal probability. Theoretical analysis and experimental results demonstrate that the WTA circuits can get comparable inference result as mean-field approximation. Taken together, our results suggest that the WTA circuit could be seen as the minimal inference unit of neuronal circuits.
Code (0)
등록된 구현이 없습니다.
Tasks
AllBayesian InferenceSimilar Papers 제목 키워드 기반
Winner-Take-All Autoencoders
In this paper, we propose a winner-take-all method for learning hierarchical sparse representations in an unsupervised fashion. We first introduce fully-connected winner-take-all autoencoders which use mini-batch statist…
AllVoltage Mode Winner-Take-All Circuit for Neuromorphic Systems
Recent advances in neuromorphic computing demonstrate on-device learning capabilities with low power consumption. One of the key learning units in these systems is the winner-take-all circuit. In this research, we propos…
Stochastic Local Winner-Takes-All Networks Enable Profound Adversarial Robustness
This work explores the potency of stochastic competition-based activations, namely Stochastic Local Winner-Takes-All (LWTA), against powerful (gradient-based) white-box and black-box adversarial attacks; we especially fo…
Adversarial AttackAdversarial DefenseAdversarial RobustnessAll+1Local Competition and Stochasticity for Adversarial Robustness in Deep Learning
This work addresses adversarial robustness in deep learning by considering deep networks with stochastic local winner-takes-all (LWTA) activations. This type of network units result in sparse representations from each mo…
Adversarial AttackAdversarial RobustnessDeep LearningA Neuromorphic Architecture for Scalable Event-Based Control
This paper introduces the ``rebound Winner-Take-All (RWTA)" motif as the basic element of a scalable neuromorphic control architecture. From the cellular level to the system level, the resulting architecture combines the…