TransSGAN: GAN based semi-superivsed learning for text classification with Transformer Encoder
Recent semi-supervised learning for text classification uses data augmentations. However, it is a time-consuming and tricky hyperparametric approach. To overcome this problem, we present GAN-based semi-supervised learning for text classification, TransSGAN, which has a simple architecture, fewer hyperparameters, and trains for less time than current SOTA models since it does not need data augmentation. By adding one transformer encoder block to Semi-Supervised GAN, we can get comparable performances with extremely few labeled data to previous SOTA models up to 1% difference and 25% better performance than baseline GAN based model. Furthermore, we provide an analysis of our model, what the generator makes, and what multi-head self-attention layer in the generator learns. Through this, we can validate that our generator makes qualified data.
Code (0)
등록된 구현이 없습니다.
Tasks
Data Augmentationtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Semi-Supervised Vision Transformers
We study the training of Vision Transformers for semi-supervised image classification. Transformers have recently demonstrated impressive performance on a multitude of supervised learning tasks. Surprisingly, we show Vis…
image-classificationImage ClassificationInductive BiasSemi-Supervised Image ClassificationText classification in shipping industry using unsupervised models and Transformer based supervised models
Obtaining labelled data in a particular context could be expensive and time consuming. Although different algorithms, including unsupervised learning, semi-supervised learning, self-learning have been adopted, the perfor…
ClassificationSelf-Learningtext-classificationText Classification+2On the Surprising Effectiveness of Transformers in Low-Labeled Video Recognition
Recently vision transformers have been shown to be competitive with convolution-based methods (CNNs) broadly across multiple vision tasks. The less restrictive inductive bias of transformers endows greater representation…
image-classificationImage ClassificationInductive BiasVideo Classification+1Transfer Learning for Fine-grained Classification Using Semi-supervised Learning and Visual Transformers
Fine-grained classification is a challenging task that involves identifying subtle differences between objects within the same category. This task is particularly challenging in scenarios where data is scarce. Visual tra…
Classificationimage-classificationImage ClassificationTransfer LearningrETF-semiSL: Semi-Supervised Learning for Neural Collapse in Temporal Data
Deep neural networks for time series must capture complex temporal patterns, to effectively represent dynamic data. Self- and semi-supervised learning methods show promising results in pre-training large models, which --…
Time Series Classification