Discontinuous phase transition of feature detection in lateral predictive coding
The brain adopts the strategy of lateral predictive coding (LPC) to construct optimal internal representations for salient features in input sensory signals to reduce the energetic cost of information transmission. Here we consider the task of distinguishing a non-Gaussian signal by LPC from $(N-1)$ Gaussian background signals of the same magnitude, which is intractable by principal component decomposition. We study the emergence of feature detection function from the perspective of statistical mechanics, and formulate a thermodynamic free energy to implement the tradeoff between energetic cost $E$ and information robustness. We define $E$ as the mean $L_1$-norm of the internal state vectors, and quantify the level of information robustness by an entropy measure $S$. We demonstrate that energy--information tradeoff may induce a discontinuous phase transition of the optimal matrix, from a very weak one with $S \approx 0$ to a functional LPC system with moderate synaptic weights in which a single unit responds selectively to the input non-Gaussian feature with high signal-to-noise ratio.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Energy--Information Trade-off Induces Continuous and Discontinuous Phase Transitions in Lateral Predictive Coding
Lateral predictive coding is a recurrent neural network which creates energy-efficient internal representations by exploiting statistical regularity in sensory inputs. Here we investigate the trade-off between informatio…
Phase Transitions in Community Detection: A Solvable Toy Model
Recently, it was shown that there is a phase transition in the community detection problem. This transition was first computed using the cavity method, and has been proved rigorously in the case of $q=2$ groups. However,…
Community DetectionA Miniature Brain Transformer: Thalamic Gating, Hippocampal Lateralization, Amygdaloid Salience, and Prefrontal Working Memory in Attention-Coupled Latent Memory
We present a miniature brain transformer architecture that extends the attention-coupled latent memory framework with four additional brain-region analogues: a thalamic relay, an amygdaloid salience module, a prefrontal …
Unsupervised feature learning from finite data by message passing: discontinuous versus continuous phase transition
Unsupervised neural network learning extracts hidden features from unlabeled training data. This is used as a pretraining step for further supervised learning in deep networks. Hence, understanding unsupervised learning …
Continuous and Discontinuous Phase Transitions in the evolution of a polygenic trait under stabilizing selective pressure
The presence of phenomena analogous to phase transition in Statistical Mechanics, has been suggested in the evolution of a polygenic trait under stabilizing selection, mutation and genetic drift. By using numerical sim…