Deep Learning of Nonnegativity-Constrained Autoencoders for Enhanced Understanding of Data
Unsupervised feature extractors are known to perform an efficient and discriminative representation of data. Insight into the mappings they perform and human ability to understand them, however, remain very limited. This is especially prominent when multilayer deep learning architectures are used. This paper demonstrates how to remove these bottlenecks within the architecture of Nonnegativity Constrained Autoencoder (NCSAE). It is shown that by using both L1 and L2 regularization that induce nonnegativity of weights, most of the weights in the network become constrained to be nonnegative thereby resulting into a more understandable structure with minute deterioration in classification accuracy. Also, this proposed approach extracts features that are more sparse and produces additional output layer sparsification. The method is analyzed for accuracy and feature interpretation on the MNIST data, the NORB normalized uniform object data, and the Reuters text categorization dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
General ClassificationL2 RegularizationText CategorizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Deep Learning of Part-based Representation of Data Using Sparse Autoencoders with Nonnegativity Constraints
We demonstrate a new deep learning autoencoder network, trained by a nonnegativity constraint algorithm (NCAE), that learns features which show part-based representation of data. The learning algorithm is based on constr…
Disentanglement with Biological Constraints: A Theory of Functional Cell Types
Neurons in the brain are often finely tuned for specific task variables. Moreover, such disentangled representations are highly sought after in machine learning. Here we mathematically prove that simple biological constr…
DisentanglementBlind and fully constrained unmixing of hyperspectral images
This paper addresses the problem of blind and fully constrained unmixing of hyperspectral images. Unmixing is performed without the use of any dictionary, and assumes that the number of constituent materials in the scene…
On the Regularization of Autoencoders
While much work has been devoted to understanding the implicit (and explicit) regularization of deep nonlinear networks in the supervised setting, this paper focuses on unsupervised learning, i.e., autoencoders are train…
Stability Analysis of Recurrent Neural Networks by IQC with Copositive Mutipliers
This paper is concerned with the stability analysis of the recurrent neural networks (RNNs) by means of the integral quadratic constraint (IQC) framework. The rectified linear unit (ReLU) is typically employed as the act…