Compressive neural representation of sparse, high-dimensional probabilities
This paper shows how sparse, high-dimensional probability distributions could be represented by neurons with exponential compression. The representation is a novel application of compressive sensing to sparse probability distributions rather than to the usual sparse signals. The compressive measurements correspond to expected values of nonlinear functions of the probabilistically distributed variables. When these expected values are estimated by sampling, the quality of the compressed representation is limited only by the quality of sampling. Since the compression preserves the geometric structure of the space of sparse probability distributions, probabilistic computation can be performed in the compressed domain. Interestingly, functions satisfying the requirements of compressive sensing can be implemented as simple perceptrons. If we use perceptrons as a simple model of feedforward computation by neurons, these results show that the mean activity of a relatively small number of neurons can accurately represent a high-dimensional joint distribution implicitly, even without accounting for any noise correlations. This comprises a novel hypothesis for how neurons could encode probabilities in the brain.
Code (0)
등록된 구현이 없습니다.
Tasks
Compressive SensingVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
On reconstruction algorithms for signals sparse in Hermite and Fourier domains
This thesis consists of original contributions in the area of digital signal processing. The reconstruction of signals sparse (highly concentrated) in various transform domains is the primary problem analyzed in the thes…
Compressive SensingDenoisingCompressive Fourier collocation methods for high-dimensional diffusion equations with periodic boundary conditions
High-dimensional Partial Differential Equations (PDEs) are a popular mathematical modelling tool, with applications ranging from finance to computational chemistry. However, standard numerical techniques for solving thes…
Compressive SensingComputational chemistryEfficient Dictionary Learning via Very Sparse Random Projections
Performing signal processing tasks on compressive measurements of data has received great attention in recent years. In this paper, we extend previous work on compressive dictionary learning by showing that more general …
ClusteringDictionary LearningFast Sublinear Sparse Representation using Shallow Tree Matching Pursuit
Sparse approximations using highly over-complete dictionaries is a state-of-the-art tool for many imaging applications including denoising, super-resolution, compressive sensing, light-field analysis, and object recognit…
Compressive SensingDenoisingImage DenoisingObject Recognition+1Compressive Sensing of Sparse Tensors
Compressive sensing (CS) has triggered enormous research activity since its first appearance. CS exploits the signal's sparsity or compressibility in a particular domain and integrates data compression and acquisition, t…
Compressive SensingData Compression