Multiscale Residual Mixture of PCA: Dynamic Dictionaries for Optimal Basis Learning
In this paper we are interested in the problem of learning an over-complete basis and a methodology such that the reconstruction or inverse problem does not need optimization. We analyze the optimality of the presented approaches, their link to popular already known techniques s.a. Artificial Neural Networks,k-means or Oja's learning rule. Finally, we will see that one approach to reach the optimal dictionary is a factorial and hierarchical approach. The derived approach lead to a formulation of a Deep Oja Network. We present results on different tasks and present the resulting very efficient learning algorithm which brings a new vision on the training of deep nets. Finally, the theoretical work shows that deep frameworks are one way to efficiently have over-complete (combinatorially large) dictionary yet allowing easy reconstruction. We thus present the Deep Residual Oja Network (DRON). We demonstrate that a recursive deep approach working on the residuals allow exponential decrease of the error w.r.t. the depth.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Multiscale Hodge Scattering Networks for Data Analysis
We propose new scattering networks for signals measured on simplicial complexes, which we call \emph{Multiscale Hodge Scattering Networks} (MHSNs). Our construction is based on multiscale basis dictionaries on simplicial…
DescriptiveMultiscale reconstruction of porous media based on multiple dictionaries learning
Digital modeling of the microstructure is important for studying the physical and transport properties of porous media. Multiscale modeling for porous media can accurately characterize macro-pores and micro-pores in a la…
Learning Better Encoding for Approximate Nearest Neighbor Search with Dictionary Annealing
We introduce a novel dictionary optimization method for high-dimensional vector quantization employed in approximate nearest neighbor (ANN) search. Vector quantization methods first seek a series of dictionaries, then ap…
Dictionary LearningGPUQuantizationJoint group and residual sparse coding for image compressive sensing
Nonlocal self-similarity and group sparsity have been widely utilized in image compressive sensing (CS). However, when the sampling rate is low, the internal prior information of degraded images may be not enough for acc…
Compressive SensingLearning Multiscale Convolutional Dictionaries for Image Reconstruction
Convolutional neural networks (CNNs) have been tremendously successful in solving imaging inverse problems. To understand their success, an effective strategy is to construct simpler and mathematically more tractable con…
Dictionary LearningImage ReconstructionMRI Reconstruction