SHOT-VAE: Semi-supervised Deep Generative Models With Label-aware ELBO Approximations
Semi-supervised variational autoencoders (VAEs) have obtained strong results, but have also encountered the challenge that good ELBO values do not always imply accurate inference results. In this paper, we investigate and propose two causes of this problem: (1) The ELBO objective cannot utilize the label information directly. (2) A bottleneck value exists and continuing to optimize ELBO after this value will not improve inference accuracy. On the basis of the experiment results, we propose SHOT-VAE to address these problems without introducing additional prior knowledge. The SHOT-VAE offers two contributions: (1) A new ELBO approximation named smooth-ELBO that integrates the label predictive loss into ELBO. (2) An approximation based on optimal interpolation that breaks the ELBO value bottleneck by reducing the margin between ELBO and the data likelihood. The SHOT-VAE achieves good performance with a 25.30% error rate on CIFAR-100 with 10k labels and reduces the error rate to 6.11% on CIFAR-10 with 4k labels.
Code (3)
Tasks
4kSemi-Supervised Image ClassificationVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SelfPrompt: Confidence-Aware Semi-Supervised Tuning for Robust Vision-Language Model Adaptation
We present SelfPrompt, a novel prompt-tuning approach for vision-language models (VLMs) in a semi-supervised learning setup. Existing methods for tuning VLMs in semi-supervised setups struggle with the negative impact of…
Language ModelingLanguage ModellingPseudo LabelWeakly-supervised LearningTarget Really Matters: Target-aware Contrastive Learning and Consistency Regularization for Few-shot Stance Detection
Stance detection aims to identify the attitude from an opinion towards a certain target. Despite the significant progress on this task, it is extremely time-consuming and budget-unfriendly to collect sufficient high-qual…
Contrastive LearningFew-Shot Stance DetectionStance DetectionSemi-Supervised Generation with Cluster-aware Generative Models
Deep generative models trained with large amounts of unlabelled data have proven to be powerful within the domain of unsupervised learning. Many real life data sets contain a small amount of labelled data points, that ar…
ClusteringGeneral ClassificationSemi-Supervised Few-Shot Classification with Deep Invertible Hybrid Models
In this paper, we propose a deep invertible hybrid model which integrates discriminative and generative learning at a latent space level for semi-supervised few-shot classification. Various tasks for classifying new spec…
ClassificationFew-Shot LearningSemi-unsupervised Learning of Human Activity using Deep Generative Models
We introduce 'semi-unsupervised learning', a problem regime related to transfer learning and zero-shot learning where, in the training data, some classes are sparsely labelled and others entirely unlabelled. Models able …
ClassificationGeneral ClassificationTime SeriesTime Series Analysis+2