Latent Neural Differential Equations for Video Generation
Generative Adversarial Networks have recently shown promise for video generation, building off of the success of image generation while also addressing a new challenge: time. Although time was analyzed in some early work, the literature has not adequately grown with temporal modeling developments. We study the effects of Neural Differential Equations to model the temporal dynamics of video generation. The paradigm of Neural Differential Equations presents many theoretical strengths including the first continuous representation of time within video generation. In order to address the effects of Neural Differential Equations, we investigate how changes in temporal models affect generated video quality. Our results give support to the usage of Neural Differential Equations as a simple replacement for older temporal generators. While keeping run times similar and decreasing parameter count, we produce a new state-of-the-art model in 64$\times$64 pixel unconditional video generation, with an Inception Score of 15.20.
Code (1)
Tasks
Unconditional Video GenerationVideo GenerationSimilar Papers 제목 키워드 기반
Identifying Latent Stochastic Differential Equations
We present a method for learning latent stochastic differential equations (SDEs) from high-dimensional time series data. Given a high-dimensional time series generated from a lower dimensional latent unknown It\^o proces…
Self-Supervised LearningTime SeriesTime Series AnalysisControllable Video Generation by Learning the Underlying Dynamical System with Neural ODE
Videos depict the change of complex dynamical systems over time in the form of discrete image sequences. Generating controllable videos by learning the dynamical system is an important yet underexplored topic in the comp…
Video GenerationSymbolic Pregression: Discovering Physical Laws from Distorted Video
We present a method for unsupervised learning of equations of motion for objects in raw and optionally distorted unlabeled video. We first train an autoencoder that maps each video frame into a low-dimensional latent spa…
regressionSymbolic RegressionAmortized Reparametrization: Efficient and Scalable Variational Inference for Latent SDEs
We consider the problem of inferring latent stochastic differential equations (SDEs) with a time and memory cost that scales independently with the amount of data, the total length of the time series, and the stiffness o…
Time SeriesVariational InferenceIntroduction to Stochastic Differential Equations for Generative Machine Learning: A Variational Perspective
The use of ordinary and stochastic differential equations has led to substantial progress in generative machine learning with applications to, for example, image, video and biomolecule generation. This paper provides a s…