Frequency Domain Transformer Networks for Video Prediction
The task of video prediction is forecasting the next frames given some previous frames. Despite much recent progress, this task is still challenging mainly due to high nonlinearity in the spatial domain. To address this issue, we propose a novel architecture, Frequency Domain Transformer Network (FDTN), which is an end-to-end learnable model that estimates and uses the transformations of the signal in the frequency domain. Experimental evaluations show that this approach can outperform some widely used video prediction methods like Video Ladder Network (VLN) and Predictive Gated Pyramids (PGP).
Code (1)
Tasks
PredictionVideo PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Motion Segmentation using Frequency Domain Transformer Networks
Self-supervised prediction is a powerful mechanism to learn representations that capture the underlying structure of the data. Despite recent progress, the self-supervised video prediction task is still challenging. One …
Motion SegmentationPredictionVideo PredictionFrequency-Aware Spatiotemporal Transformers for Video Inpainting Detection
In this paper, we propose a frequency-aware spatiotemporal transformers for deep In this paper, we propose a Frequency-Aware Spatiotemporal Transformer (FAST) for video inpainting detection, which aims to simultaneou…
DecoderVideo InpaintingRethinking Urban Mobility Prediction: A Super-Multivariate Time Series Forecasting Approach
Long-term urban mobility predictions play a crucial role in the effective management of urban facilities and services. Conventionally, urban mobility data has been structured as spatiotemporal videos, treating longitude …
Multivariate Time Series ForecastingTime SeriesTime Series ForecastingVideo PredictionLearning Spatiotemporal Frequency-Transformer for Low-Quality Video Super-Resolution
Video Super-Resolution (VSR) aims to restore high-resolution (HR) videos from low-resolution (LR) videos. Existing VSR techniques usually recover HR frames by extracting pertinent textures from nearby frames with known d…
Super-ResolutionVideo EnhancementVideo Super-ResolutionSemantic Prediction: Which One Should Come First, Recognition or Prediction?
The ultimate goal of video prediction is not forecasting future pixel-values given some previous frames. Rather, the end goal of video prediction is to discover valuable internal representations from the vast amount of a…
Decision MakingPredictionSemantic CompositionVideo Prediction