Better Mixing via Deep Representations
It has previously been hypothesized, and supported with some experimental evidence, that deeper representations, when well trained, tend to do a better job at disentangling the underlying factors of variation. We study the following related conjecture: better representations, in the sense of better disentangling, can be exploited to produce faster-mixing Markov chains. Consequently, mixing would be more efficient at higher levels of representation. To better understand why and how this is happening, we propose a secondary conjecture: the higher-level samples fill more uniformly the space they occupy and the high-density manifolds tend to unfold when represented at higher levels. The paper discusses these hypotheses and tests them experimentally through visualization and measurements of mixing and interpolating between samples.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Hard Negative Mixing for Contrastive Learning
Contrastive learning has become a key component of self-supervised learning approaches for computer vision. By learning to embed two augmented versions of the same image close to each other and to push the embeddings of …
Contrastive LearningData AugmentationInstance Segmentationobject-detection+3Minority Positive Sampling for Switching Points - an Anecdote for the Code-Mixing Language Modeling
Code-Mixing (CM) or language mixing is a social norm in multilingual societies. CM is quite prevalent in social media conversations in multilingual regions like - India, Europe, Canada and Mexico. In this paper, we explo…
Language ModelingLanguage ModellingDeep Hyperspectral Unmixing using Transformer Network
Currently, this paper is under review in IEEE. Transformers have intrigued the vision research community with their state-of-the-art performance in natural language processing. With their superior performance, transforme…
DecoderHyperspectral Image ClassificationHyperspectral Unmixingimage-classification+1TransformMix: Learning Transformation and Mixing Strategies from Data
Data augmentation improves the generalization power of deep learning models by synthesizing more training samples. Sample-mixing is a popular data augmentation approach that creates additional data by combining existing …
Data AugmentationKnowledge Distillationobject-detectionObject Detection+1Hyperspectral Blind Unmixing using a Double Deep Image Prior
With the rise of machine learning, hyperspectral image (HSI) unmixing problems have been tackled using learning-based methods. However, physically meaningful unmixing results are not guaranteed without proper guidance. I…
Hyperspectral UnmixingImage GenerationInterpretable Machine LearningPhysics-informed machine learning