Over-complete representations on recurrent neural networks can support persistent percepts

2010-12-01 · NeurIPS 2010 12 · Shaul Druckmann, Dmitri B. Chklovskii

A striking aspect of cortical neural networks is the divergence of a relatively small number of input channels from the peripheral sensory apparatus into a large number of cortical neurons, an over-complete representation strategy. Cortical neurons are then connected by a sparse network of lateral synapses. Here we propose that such architecture may increase the persistence of the representation of an incoming stimulus, or a percept. We demonstrate that for a family of networks in which the receptive field of each neuron is re-expressed by its outgoing connections, a represented percept can remain constant despite changing activity. We term this choice of connectivity REceptive FIeld REcombination (REFIRE) networks. The sparse REFIRE network may serve as a high-dimensional integrator and a biologically plausible model of the local cortical circuit.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Policy Iteration with Human Feedback: Bringing Post-Training RL to In-context Learning

2026-08-17 · Minh-Ha Nguyen, Cathy Shyr arxiv

Generative pretraining established reusable task representations; later work on language-based task conditioning and in-context learning showed that a fixed model could adapt its behavior from instructions and demonstrat…

Persistent Recurrent Memory Between Transformer Layers - Improves Language Model Generalization

2026-09-15 · Eduardo Novaes Hering arxiv

We introduce a simple architectural modification to decoder-only transformers: a persistent recurrent state that observes hidden representations via cross-attention, updates itself through a GRU, and modulates subsequent…

Beyond Episodic AI: Cognitive Field Networks for Biologically Inspired Persistent Cognition

2026-09-15 · Byung Gyu Chae arxiv

Cognitive Field Theory (CFT) proposes that cognition arises from memory-dressed collective dynamics that generate a persistent macroscopic cognitive field. Here we develop a Cognitive Field Network (CFN), a recurrent Tra…

Persistent Gaussian Perturbations Prevent Oversmoothing in Recurrent Graph Neural Networks

2026-07-30 · Mostafa Haghir Chehreghani arxiv

Oversmoothing is a fundamental limitation of deep graph neural networks (GNNs), where repeated message passing causes node representations to become increasingly similar, eventually collapsing toward a low-dimensional su…

Graph Neural Network

Bootstrap Theory of Representational Emergence (TBER): Explanatory Insufficiency, Transition Regimes, and the Emergence of New Representational Levels

2026-06-05 · Jacques Raynal, Pierre Slangen, Elsa Raynal, Jacques Margerit arxiv

Representation learning is central to modern machine learning, yet most research focuses on optimizing representations after a framework has been selected. The Bootstrap Theory of Representational Emergence (TBER) addres…

Representation Learning Anomaly Detection