Learning Implicit Generative Models by Teaching Explicit Ones
Implicit generative models are difficult to train as no explicit density functions are defined. Generative adversarial nets (GANs) present a minimax framework to train such models, which however can suffer from mode collapse due to the nature of the JS-divergence. This paper presents a learning by teaching (LBT) approach to learning implicit models, which intrinsically avoids the mode collapse problem by optimizing a KL-divergence rather than the JS-divergence in GANs. In LBT, an auxiliary density estimator is introduced to fit the implicit model's distribution while the implicit model teaches the density estimator to match the data distribution. LBT is formulated as a bilevel optimization problem, whose optimal generator matches the true data distribution. LBT can be naturally integrated with GANs to derive a hybrid LBT-GAN that enjoys complimentary benefits. Finally, we present a stochastic gradient ascent algorithm with unrolling to solve the challenging learning problems. Experimental results demonstrate the effectiveness of our method.
Code (0)
등록된 구현이 없습니다.
Tasks
Bilevel OptimizationRolling Shutter CorrectionSimilar Papers 제목 키워드 기반
Learning to Generate Synthetic Training Data using Gradient Matching and Implicit Differentiation
Using huge training datasets can be costly and inconvenient. This article explores various data distillation techniques that can reduce the amount of data required to successfully train deep networks. Inspired by recent …
image-classificationImage ClassificationProsody as a Teaching Signal for Agent Learning: Exploratory Studies and Algorithmic Implications
Agent learning from human interaction often relies on explicit signals, but implicit social cues, such as prosody in speech, could provide valuable information for more effective learning. This paper advocates for the in…
Atari Gamesreinforcement-learningReinforcement LearningGuidance and Teaching Network for Video Salient Object Detection
Owing to the difficulties of mining spatial-temporal cues, the existing approaches for video salient object detection (VSOD) are limited in understanding complex and noisy scenarios, and often fail in inferring prominent…
GPUObjectobject-detectionObject Detection+2Leap-Of-Thought: Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge
To what extent can a neural network systematically reason over symbolic facts? Evidence suggests that large pre-trained language models (LMs) acquire some reasoning capacity, but this ability is difficult to control. Rec…
World KnowledgeNext3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatars
3D-aware generative adversarial networks (GANs) synthesize high-fidelity and multi-view-consistent facial images using only collections of single-view 2D imagery. Towards fine-grained control over facial attributes, rece…
Face Model