Challenging the Semi-Supervised VAE Framework for Text Classification
Semi-Supervised Variational Autoencoders (SSVAEs) are widely used models for data efficient learning. In this paper, we question the adequacy of the standard design of sequence SSVAEs for the task of text classification as we exhibit two sources of overcomplexity for which we provide simplifications. These simplifications to SSVAEs preserve their theoretical soundness while providing a number of practical advantages in the semi-supervised setup where the result of training is a text classifier. These simplifications are the removal of (i) the Kullback-Liebler divergence from its objective and (ii) the fully unobserved latent variable from its probabilistic model. These changes relieve users from choosing a prior for their latent variables, make the model smaller and faster, and allow for a better flow of information into the latent variables. We compare the simplified versions to standard SSVAEs on 4 text classification tasks. On top of the above-mentioned simplification, experiments show a speed-up of 26%, while keeping equivalent classification scores. The code to reproduce our experiments is public.
Code (1)
Tasks
Classificationtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Interpretable Operational Risk Classification with Semi-Supervised Variational Autoencoder
Operational risk management is one of the biggest challenges nowadays faced by financial institutions. There are several major challenges of building a text classification system for automatic operational risk prediction…
ClassificationGeneral ClassificationManagementSemi-Supervised Text Classification+3Universal Semi-Supervised Learning for Medical Image Classification
Semi-supervised learning (SSL) has attracted much attention since it reduces the expensive costs of collecting adequate well-labeled training data, especially for deep learning methods. However, traditional SSL is built …
ClassificationDomain Adaptationimage-classificationImage Classification+2Understanding Social Support Needs in Questions: A Hybrid Approach Integrating Semi-Supervised Learning and LLM-based Data Augmentation
Patients are increasingly turning to online health Q&A communities for social support to improve their well-being. However, when this support received does not align with their specific needs, it may prove ineffective or…
Data AugmentationSemi-supervised Domain Adaptation on Graphs with Contrastive Learning and Minimax Entropy
Label scarcity in a graph is frequently encountered in real-world applications due to the high cost of data labeling. To this end, semi-supervised domain adaptation (SSDA) on graphs aims to leverage the knowledge of a la…
Contrastive LearningDomain AdaptationNode ClassificationSemi-supervised Domain AdaptationColour augmentation for improved semi-supervised semantic segmentation
Consistency regularization describes a class of approaches that have yielded state-of-the-art results for semi-supervised classification. While semi-supervised semantic segmentation proved to be more challenging, a numbe…
ClassificationSegmentationSelf-Supervised LearningSemantic Segmentation+1