Decentralized Complete Dictionary Learning via $\ell^{4}$-Norm Maximization
With the rapid development of information technologies, centralized data processing is subject to many limitations, such as computational overheads, communication delays, and data privacy leakage. Decentralized data processing over networked terminal nodes becomes an important technology in the era of big data. Dictionary learning is a powerful representation learning method to exploit the low-dimensional structure from the high-dimensional data. By exploiting the low-dimensional structure, the storage and the processing overhead of data can be effectively reduced. In this paper, we propose a novel decentralized complete dictionary learning algorithm, which is based on $\ell^{4}$-norm maximization. Compared with existing decentralized dictionary learning algorithms, comprehensive numerical experiments show that the novel algorithm has significant advantages in terms of per-iteration computational complexity, communication cost, and convergence rate in many scenarios. Moreover, a rigorous theoretical analysis shows that the dictionaries learned by the proposed algorithm can converge to the one learned by a centralized dictionary learning algorithm at a linear rate with high probability under certain conditions.
Code (0)
등록된 구현이 없습니다.
Tasks
Dictionary LearningRepresentation LearningSimilar Papers 제목 키워드 기반
Complete Dictionary Learning via $\ell_p$-norm Maximization
Dictionary learning is a classic representation learning method that has been widely applied in signal processing and data analytics. In this paper, we investigate a family of $\ell_p$-norm ($p>2,p \in \mathbb{N}$) maxim…
Computational EfficiencyDictionary LearningRepresentation LearningComplete Dictionary Learning via $\ell^4$-Norm Maximization over the Orthogonal Group
This paper considers the fundamental problem of learning a complete (orthogonal) dictionary from samples of sparsely generated signals. Most existing methods solve the dictionary (and sparse representations) based on heu…
Dictionary LearningDeep Residual Autoencoders for Expectation Maximization-inspired Dictionary Learning
We introduce a neural-network architecture, termed the constrained recurrent sparse autoencoder (CRsAE), that solves convolutional dictionary learning problems, thus establishing a link between dictionary learning and ne…
DenoisingDictionary LearningImage DenoisingExact Sparse Orthogonal Dictionary Learning
Over the past decade, learning a dictionary from input images for sparse modeling has been one of the topics which receive most research attention in image processing and compressed sensing. Most existing dictionary lear…
compressed sensingDenoisingDictionary LearningAlternating minimization for dictionary learning with random initialization
We present theoretical guarantees for an alternating minimization algorithm for the dictionary learning/sparse coding problem. The dictionary learning problem is to factorize vector samples $y^{1},y^{2},\ldots, y^{n}$ in…
Dictionary Learning