MSTCGAN: Multiscale time conditional generative adversarial network for long-term satellite image sequence prediction
Satellite image sequence prediction is a crucial and challenging task. Previous studies leverage optical flow methods or existing deep learning methods on spatial–temporal sequence models for the task. However, they suffer from either oversimplified model assumptions or blurry predictions and sequential error accumulation issue, for a long-term forecast requirement. In this article, we propose a novel multiscale time conditional generative adversarial network (MSTCGAN). To address the sequential error accumulation issue, MSTCGAN adopts a parallel prediction framework to produce the future image sequences by a one-hot time condition input. In addition, a powerful multiscale generator is designed with the multihead axial attention, which helps to carefully preserve the fine-grained details for appearance consistency. Moreover, we develop a temporal discriminator to address the blurry issue and maintain the motion consistency in prediction. Extensive experiments have been conducted on the FengYun-4A satellite dataset, and the results demonstrate the effectiveness and superiority of the proposed method over state-of-the-art approaches.
Code (1)
Tasks
Generative Adversarial NetworkOptical Flow EstimationPredictionSimilar Papers 제목 키워드 기반
Downhole Track Detection via Multiscale Conditional Generative Adversarial Nets
Frequent mine disasters cause a large number of casualties and property losses. Autonomous driving is a fundamental measure for solving this problem, and track detection is one of the key technologies for computer vision…
Autonomous DrivingGenerative Adversarial NetworkGANs and Closures: Micro-Macro Consistency in Multiscale Modeling
Sampling the phase space of molecular systems -- and, more generally, of complex systems effectively modeled by stochastic differential equations -- is a crucial modeling step in many fields, from protein folding to mate…
Dimensionality ReductionProtein FoldingAdversarial Multiscale Feature Learning for Overlapping Chromosome Segmentation
Chromosome karyotype analysis is of great clinical importance in the diagnosis and treatment of diseases, especially for genetic diseases. Since manual analysis is highly time and effort consuming, computer-assisted auto…
Generative Adversarial NetworkSegmentationThe efficacy and generalizability of conditional GANs for posterior inference in physics-based inverse problems
In this work, we train conditional Wasserstein generative adversarial networks to effectively sample from the posterior of physics-based Bayesian inference problems. The generator is constructed using a U-Net architectur…
Bayesian InferenceExtreme Image Coding via Multiscale Autoencoders With Generative Adversarial Optimization
We propose a MultiScale AutoEncoder(MSAE) based extreme image compression framework to offer visually pleasing reconstruction at a very low bitrate. Our method leverages the "priors" at different resolution scale to impr…
Generative Adversarial NetworkImage Compression