Multiscale Sparse Microcanonical Models
We study approximations of non-Gaussian stationary processes having long range correlations with microcanonical models. These models are conditioned by the empirical value of an energy vector, evaluated on a single realization. Asymptotic properties of maximum entropy microcanonical and macrocanonical processes and their convergence to Gibbs measures are reviewed. We show that the Jacobian of the energy vector controls the entropy rate of microcanonical processes. Sampling maximum entropy processes through MCMC algorithms require too many operations when the number of constraints is large. We define microcanonical gradient descent processes by transporting a maximum entropy measure with a gradient descent algorithm which enforces the energy conditions. Convergence and symmetries are analyzed. Approximations of non-Gaussian processes with long range interactions are defined with multiscale energy vectors computed with wavelet and scattering transforms. Sparsity properties are captured with $\bf l^1$ norms. Approximations of Gaussian, Ising and point processes are studied, as well as image and audio texture synthesis.
Code (0)
등록된 구현이 없습니다.
Tasks
Gaussian ProcessesPoint ProcessesTexture SynthesisSimilar Papers 제목 키워드 기반
Circumventing spin glass traps by microcanonical spontaneous symmetry breaking
The planted p-spin interaction model is a paradigm of random-graph systems possessing both a ferromagnetic phase and a disordered phase with the latter splitting into many spin glass states at low temperatures. Conventio…
Can Microcanonical Langevin Dynamics Leverage Mini-Batch Gradient Noise?
Scaling inference methods such as Markov chain Monte Carlo to high-dimensional models remains a central challenge in Bayesian deep learning. A promising recent proposal, microcanonical Langevin Monte Carlo, has shown sta…
Bayesian InferenceMean-Field Microcanonical Gradient Descent
Microcanonical gradient descent is a sampling procedure for energy-based models allowing for efficient sampling of distributions in high dimension. It works by transporting samples from a high-entropy distribution, such …
Time SeriesHyper-Parameter Selection in Convolutional Neural Networks Using Microcanonical Optimization Algorithm
The success of Convolutional Neural Networks is highly dependent on the selected architecture and the hyper-parameters. The need for the automatic design of the networks is especially important for complex architectures …
Microcanonical Langevin Ensembles: Advancing the Sampling of Bayesian Neural Networks
Despite recent advances, sampling-based inference for Bayesian Neural Networks (BNNs) remains a significant challenge in probabilistic deep learning. While sampling-based approaches do not require a variational distribut…
NavigateProbabilistic Deep LearningUncertainty Quantification