Large Scale Adversarial Representation Learning
Adversarially trained generative models (GANs) have recently achieved compelling image synthesis results. But despite early successes in using GANs for unsupervised representation learning, they have since been superseded by approaches based on self-supervision. In this work we show that progress in image generation quality translates to substantially improved representation learning performance. Our approach, BigBiGAN, builds upon the state-of-the-art BigGAN model, extending it to representation learning by adding an encoder and modifying the discriminator. We extensively evaluate the representation learning and generation capabilities of these BigBiGAN models, demonstrating that these generation-based models achieve the state of the art in unsupervised representation learning on ImageNet, as well as in unconditional image generation. Pretrained BigBiGAN models -- including image generators and encoders -- are available on TensorFlow Hub (https://tfhub.dev/s?publisher=deepmind&q=bigbigan).
Code (4)
Tasks
Contrastive LearningImage GenerationRepresentation LearningSelf-Supervised Image ClassificationSemi-Supervised Image ClassificationUnconditional Image GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Large-Scale Adversarial Training for Vision-and-Language Representation Learning
We present VILLA, the first known effort on large-scale adversarial training for vision-and-language (V+L) representation learning. VILLA consists of two training stages: (i) task-agnostic adversarial pre-training; follo…
Image-text RetrievalQuestion AnsweringReferring ExpressionReferring Expression Comprehension+8MHTN: Modal-adversarial Hybrid Transfer Network for Cross-modal Retrieval
Cross-modal retrieval has drawn wide interest for retrieval across different modalities of data. However, existing methods based on DNN face the challenge of insufficient cross-modal training data, which limits the train…
Cross-Modal RetrievalRepresentation LearningRetrievalTransfer LearningBetter Representations via Adversarial Training in Pre-Training: A Theoretical Perspective
Pre-training is known to generate universal representations for downstream tasks in large-scale deep learning such as large language models. Existing literature, e.g., \cite{kim2020adversarial}, empirically observe that …
Adversarial RobustnessContrastive LearningvalidCalibrated Domain-Invariant Learning for Highly Generalizable Large Scale Re-Identification
Many real-world applications, such as city-scale traffic monitoring and control, requires large-scale re-identification. However, previous ReID methods often failed to address two limitations in existing ReID benchmarks,…
Efficient Adversarial Contrastive Learning via Robustness-Aware Coreset Selection
Adversarial contrastive learning (ACL) does not require expensive data annotations but outputs a robust representation that withstands adversarial attacks and also generalizes to a wide range of downstream tasks. However…
Contrastive Learning