AdaFNIO: Adaptive Fourier Neural Interpolation Operator for video frame interpolation
We present, AdaFNIO - Adaptive Fourier Neural Interpolation Operator, a neural operator-based architecture to perform video frame interpolation. Current deep learning based methods rely on local convolutions for feature learning and suffer from not being scale-invariant, thus requiring training data to be augmented through random flipping and re-scaling. On the other hand, AdaFNIO, learns the features in the frames, independent of input resolution, through token mixing and global convolution in the Fourier space or the spectral domain by using Fast Fourier Transform (FFT). We show that AdaFNIO can produce visually smooth and accurate results. To evaluate the visual quality of our interpolated frames, we calculate the structural similarity index (SSIM) and Peak Signal to Noise Ratio (PSNR) between the generated frame and the ground truth frame. We provide the quantitative performance of our model on Vimeo-90K dataset, DAVIS, UCF101 and DISFA+ dataset.
Code (1)
Tasks
SSIMVideo Frame InterpolationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Modulated Adaptive Fourier Neural Operators for Temporal Interpolation of Weather Forecasts
Weather and climate data are often available at limited temporal resolution, either due to storage limitations, or in the case of weather forecast models based on deep learning, their inherently long time steps. The coar…
Weather ForecastingFB-Diff: Fourier Basis-guided Diffusion for Temporal Interpolation of 4D Medical Imaging
The temporal interpolation task for 4D medical imaging, plays a crucial role in clinical practice of respiratory motion modeling. Following the simplified linear-motion hypothesis, existing approaches adopt optical flow-…
Optimal Learning Rates for Regularized Least-Squares with a Fourier Capacity Condition
We derive minimax adaptive rates for a new, broad class of Tikhonov-regularized learning problems in Hilbert scales under general source conditions. Our analysis does not require the regression function to be contained i…
regressionResolution-Invariant Image Classification based on Fourier Neural Operators
In this paper we investigate the use of Fourier Neural Operators (FNOs) for image classification in comparison to standard Convolutional Neural Networks (CNNs). Neural operators are a discretization-invariant generalizat…
Classificationimage-classificationImage ClassificationNon-equispaced Fourier Neural Solvers for PDEs
Solving partial differential equations is difficult. Recently proposed neural resolution-invariant models, despite their effectiveness and efficiency, usually require equispaced spatial points of data. However, sampling …