INFORMATION MAXIMIZATION AUTO-ENCODING
We propose the Information Maximization Autoencoder (IMAE), an information theoretic approach to simultaneously learn continuous and discrete representations in an unsupervised setting. Unlike the Variational Autoencoder framework, IMAE starts from a stochastic encoder that seeks to map each input data to a hybrid discrete and continuous representation with the objective of maximizing the mutual information between the data and their representations. A decoder is included to approximate the posterior distribution of the data given their representations, where a high fidelity approximation can be achieved by leveraging the informative representations. We show that the proposed objective is theoretically valid and provides a principled framework for understanding the tradeoffs regarding informativeness of each representation factor, disentanglement of representations, and decoding quality.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderDisentanglementInformativenessvalidMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Auto-Encoding Total Correlation Explanation
Advances in unsupervised learning enable reconstruction and generation of samples from complex distributions, but this success is marred by the inscrutability of the representations learned. We propose an information-the…
DisentanglementRepresentation LearningUncertainty Autoencoders: Learning Compressed Representations via Variational Information Maximization
Compressed sensing techniques enable efficient acquisition and recovery of sparse, high-dimensional data signals via low-dimensional projections. In this work, we propose Uncertainty Autoencoders, a learning framework fo…
compressed sensingDimensionality ReductionRepresentation LearningDeep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection
Unsupervised anomaly detection on multi- or high-dimensional data is of great importance in both fundamental machine learning research and industrial applications, for which density estimation lies at the core. Although …
Anomaly DetectionDensity EstimationDimensionality ReductionUnsupervised Anomaly Detection+5AutoSamp: Autoencoding k-space Sampling via Variational Information Maximization for 3D MRI
Accelerated MRI protocols routinely involve a predefined sampling pattern that undersamples the k-space. Finding an optimal pattern can enhance the reconstruction quality, however this optimization is a challenging task.…
Anatomycompressed sensingDecoderVisualizing and Controlling Cortical Responses Using Voxel-Weighted Activation Maximization
Deep neural networks (DNNs) trained on visual tasks develop feature representations that resemble those in the human visual system. Although DNN-based encoding models can accurately predict brain responses to visual stim…