Language Modeling with Generative AdversarialNetworks
Generative Adversarial Networks (GANs) have been promising in the field of image generation, however, they have been hard to train for language generation. GANs were originally designed to output differentiable values, so discrete language generation is challenging for them which causes high levels of instability in training GANs. Consequently, past work has resorted to pre-training with maximum-likelihood or training GANs without pre-training with a WGAN objective with a gradient penalty. In this study, we present a comparison of those approaches. Furthermore, we present the results of some experiments that indicate better training and convergence of Wasserstein GANs (WGANs) when a weaker regularization term is enforcing the Lipschitz constraint.
Code (0)
등록된 구현이 없습니다.
Tasks
Image GenerationLanguage ModelingLanguage ModellingText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Image Generation and Editing with Variational Info Generative AdversarialNetworks
Recently there has been an enormous interest in generative models for images in deep learning. In pursuit of this, Generative Adversarial Networks (GAN) and Variational Auto-Encoder (VAE) have surfaced as two most promin…
Image GenerationLearning disconnected manifolds: a no GANs land
Typical architectures of Generative AdversarialNetworks make use of a unimodal latent distribution transformed by a continuous generator. Consequently, the modeled distribution always has connected support which is cumbe…
TWIST-GAN: Towards Wavelet Transform and Transferred GAN for Spatio-Temporal Single Image Super Resolution
Single Image Super-resolution (SISR) produces high-resolution images with fine spatial resolutions from aremotely sensed image with low spatial resolution. Recently, deep learning and generative adversarial networks(GANs…
Generative Adversarial NetworkImage Super-ResolutionSuper-ResolutionA Generative Parser with a Discriminative Recognition Algorithm
Generative models defining joint distributions over parse trees and sentences are useful for parsing and language modeling, but impose restrictions on the scope of features and are often outperformed by discriminative mo…
Constituency ParsingDecoderLanguage ModelingLanguage Modelling+1Language Modeling for Code-Switching: Evaluation, Integration of Monolingual Data, and Discriminative Training
We focus on the problem of language modeling for code-switched language, in the context of automatic speech recognition (ASR). Language modeling for code-switched language is challenging for (at least) three reasons: (1)…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Language ModelingLanguage Modelling+2