Dual-MTGAN: Stochastic and Deterministic Motion Transfer for Image-to-Video Synthesis
Generating videos with content and motion variations is a challenging task in computer vision. While the recent development of GAN allows video generation from latent representations, it is not easy to produce videos with particular content of motion patterns of interest. In this paper, we propose Dual Motion Transfer GAN (Dual-MTGAN), which takes image and video data as inputs while learning disentangled content and motion representations. Our Dual-MTGAN is able to perform deterministic motion transfer and stochastic motion generation. Based on a given image, the former preserves the input content and transfers motion patterns observed from another video sequence, and the latter directly produces videos with plausible yet diverse motion patterns based on the input image. The proposed model is trained in an end-to-end manner, without the need to utilize pre-defined motion features like pose or facial landmarks. Our quantitative and qualitative results would confirm the effectiveness and robustness of our model in addressing such conditioned image-to-video tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Motion GenerationVideo GenerationSimilar Papers 제목 키워드 기반
Discovering mesoscopic descriptions of collective movement with neural stochastic modelling
Collective motion is an ubiquitous phenomenon in nature, inspiring engineers, physicists and mathematicians to develop mathematical models and bio-inspired designs. Collective motion at small to medium group sizes ($\sim…
Optimal Dividends in the Dual Risk Model under a Stochastic Interest Rate
Optimal dividend strategy in dual risk model is well studied in the literatures. But to the best of our knowledge, all the previous works assumes deterministic interest rate. In this paper, we study the optimal dividends…
Fractional Stochastic Neural Networks
In this paper, we develop a fractional stochastic neural network with residual dynamics driven by fractional Brownian motion. By introducing a discrete stochastic maximum principle for the network, we construct the corre…
Image ClassificationInference-time Stochastic Refinement of GRU-Normalizing Flow for Real-time Video Motion Transfer
Real-time video motion transfer applications such as immersive gaming and vision-based anomaly detection require accurate yet diverse future predictions to support realistic synthesis and robust downstream decision makin…
Time Series ForecastingAnomaly DetectionDecision MakingMulti-Label Clinical Time-Series Generation via Conditional GAN
In recent years, deep learning has been successfully adopted in a wide range of applications related to electronic health records (EHRs) such as representation learning and clinical event prediction. However, due to priv…
Representation LearningTime SeriesTime Series AnalysisTime Series Generation