Deep Matching Autoencoders
Increasingly many real world tasks involve data in multiple modalities or views. This has motivated the development of many effective algorithms for learning a common latent space to relate multiple domains. However, most existing cross-view learning algorithms assume access to paired data for training. Their applicability is thus limited as the paired data assumption is often violated in practice: many tasks have only a small subset of data available with pairing annotation, or even no paired data at all. In this paper we introduce Deep Matching Autoencoders (DMAE), which learn a common latent space and pairing from unpaired multi-modal data. Specifically we formulate this as a cross-domain representation learning and object matching problem. We simultaneously optimise parameters of representation learning auto-encoders and the pairing of unpaired multi-modal data. This framework elegantly spans the full regime from fully supervised, semi-supervised, and unsupervised (no paired data) multi-modal learning. We show promising results in image captioning, and on a new task that is uniquely enabled by our methodology: unsupervised classifier learning.
Code (0)
등록된 구현이 없습니다.
Tasks
Image CaptioningRepresentation LearningSimilar Papers 제목 키워드 기반
Manifold-Matching Autoencoders
We study a simple unsupervised regularization scheme for autoencoders called Manifold-Matching (MMAE): we align the pairwise distances in the latent space to those of the input data space by minimizing mean squared error…
An Unsupervised Model with Attention Autoencoders for Question Retrieval
Question retrieval is a crucial subtask for community question answering. Previous research focus on supervised models which depend heavily on training data and manual feature engineering. In this paper, we propose a nov…
Community Question AnsweringFeature EngineeringQuestion AnsweringRetrievalTheoretical Insights into the Use of Structural Similarity Index In Generative Models and Inferential Autoencoders
Generative models and inferential autoencoders mostly make use of $\ell_2$ norm in their optimization objectives. In order to generate perceptually better images, this short paper theoretically discusses how to use Struc…
Dimensionality ReductionImage GenerationImage Quality AssessmentSSIMInformation Theoretic-Learning Auto-Encoder
We propose Information Theoretic-Learning (ITL) divergence measures for variational regularization of neural networks. We also explore ITL-regularized autoencoders as an alternative to variational autoencoding bayes, adv…
Evaluating Sparse Autoencoders: From Shallow Design to Matching Pursuit
Sparse autoencoders (SAEs) have recently become central tools for interpretability, leveraging dictionary learning principles to extract sparse, interpretable features from neural representations whose underlying structu…
Dictionary Learning