Learning to navigate image manifolds induced by generative adversarial networks for unsupervised video generation
In this work, we introduce a two-step framework for generative modeling of temporal data. Specifically, the generative adversarial networks (GANs) setting is employed to generate synthetic scenes of moving objects. To do so, we propose a two-step training scheme within which: a generator of static frames is trained first. Afterwards, a recurrent model is trained with the goal of providing a sequence of inputs to the previously trained frames generator, thus yielding scenes which look natural. The adversarial setting is employed in both training steps. However, with the aim of avoiding known training instabilities in GANs, a multiple discriminator approach is used to train both models. Results in the studied video dataset indicate that, by employing such an approach, the recurrent part is able to learn how to coherently navigate the image manifold induced by the frames generator, thus yielding more natural-looking scenes.
Code (1)
Tasks
NavigateVideo GenerationSimilar Papers 제목 키워드 기반
Manifold Learning and Alignment with Generative Adversarial Networks
We present a generative adversarial network (GAN) that conducts manifold learning and alignment (MLA): A task to learn the multi-manifold structure underlying data and to align those manifolds without any correspondence …
DisentanglementGenerative Adversarial NetworkQuantifying Manifolds: Do the manifolds learned by Generative Adversarial Networks converge to the real data manifold
This paper presents our experiments to quantify the manifolds learned by ML models (in our experiment, we use a GAN model) as they train. We compare the manifolds learned at each epoch to the real manifolds representing …
MMGAN: Manifold Matching Generative Adversarial Network
It is well-known that GANs are difficult to train, and several different techniques have been proposed in order to stabilize their training. In this paper, we propose a novel training method called manifold-matching, and…
Generative Adversarial NetworkManifold-valued Image Generation with Wasserstein Generative Adversarial Nets
Generative modeling over natural images is one of the most fundamental machine learning problems. However, few modern generative models, including Wasserstein Generative Adversarial Nets (WGANs), are studied on manifold-…
Image GenerationMask-Guided Discovery of Semantic Manifolds in Generative Models
Advances in the realm of Generative Adversarial Networks (GANs) have led to architectures capable of producing amazingly realistic images such as StyleGAN2, which, when trained on the FFHQ dataset, generates images of hu…
Disentanglement