Encoding Binary Concepts in the Latent Space of Generative Models for Enhancing Data Representation
Binary concepts are empirically used by humans to generalize efficiently. And they are based on Bernoulli distribution which is the building block of information. These concepts span both low-level and high-level features such as "large vs small" and "a neuron is active or inactive". Binary concepts are ubiquitous features and can be used to transfer knowledge to improve model generalization. We propose a novel binarized regularization to facilitate learning of binary concepts to improve the quality of data generation in autoencoders. We introduce a binarizing hyperparameter $r$ in data generation process to disentangle the latent space symmetrically. We demonstrate that this method can be applied easily to existing variational autoencoder (VAE) variants to encourage symmetric disentanglement, improve reconstruction quality, and prevent posterior collapse without computation overhead. We also demonstrate that this method can boost existing models to learn more transferable representations and generate more representative samples for the input distribution which can alleviate catastrophic forgetting using generative replay under continual learning settings.
Code (1)
Tasks
Continual LearningDisentanglementSimilar Papers 제목 키워드 기반
BinPlay: A Binary Latent Autoencoder for Generative Replay Continual Learning
We introduce a binary latent space autoencoder architecture to rehearse training samples for the continual learning of neural networks. The ability to extend the knowledge of a model with new data without forgetting prev…
Continual LearningEmergence of Latent Binary Encoding in Deep Neural Network Classifiers
We investigate the emergence of binary encoding within the latent space of deep-neural-network classifiers. Such binary encoding is induced by the introduction of a linear penultimate layer, which employs during training…
CausalEGM: a general causal inference framework by encoding generative modeling
Although understanding and characterizing causal effects have become essential in observational studies, it is challenging when the confounders are high-dimensional. In this article, we develop a general framework $\text…
Causal InferenceDecoderLatent graph encoding of multimodal neuroimaging features with generative AI architectures
While generative models enable encoding of complex neuroimaging data for feature generation and reconstruction, developing optimal architectural frameworks with appropriate encoding and latent space processes is crucial …
Generative Binary Memory: Pseudo-Replay Class-Incremental Learning on Binarized Embeddings
In dynamic environments where new concepts continuously emerge, Deep Neural Networks (DNNs) must adapt by learning new classes while retaining previously acquired ones. This challenge is addressed by Class-Incremental Le…
class-incremental learningClass Incremental LearningIncremental Learning