ViSIR: Vision Transformer Single Image Reconstruction Method for Earth System Models
Purpose: Earth system models (ESMs) integrate the interactions of the atmosphere, ocean, land, ice, and biosphere to estimate the state of regional and global climate under a wide variety of conditions. The ESMs are highly complex, and thus, deep neural network architectures are used to model the complexity and store the down-sampled data. In this paper, we propose the Vision Transformer Sinusoidal Representation Networks (ViSIR) to improve the single image SR (SR) reconstruction task for the ESM data. Methods: ViSIR combines the SR capability of Vision Transformers (ViT) with the high-frequency detail preservation of the Sinusoidal Representation Network (SIREN) to address the spectral bias observed in SR tasks. Results: The ViSIR outperforms ViT by 4.1 dB, SIREN by 7.5 dB, and SR-Generative Adversarial (SR-GANs) by 7.1dB PSNR on average for three different measurements. Conclusion: The proposed ViSIR is evaluated and compared with state-of-the-art methods. The results show that the proposed algorithm is outperforming other methods in terms of Mean Square Error(MSE), Peak-Signal-to-Noise-Ratio(PSNR), and Structural Similarity Index Measure(SSIM).
Code (0)
등록된 구현이 없습니다.
Tasks
Image ReconstructionSSIMMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restoration
Vision-centric perception systems often struggle with unpredictable and coupled weather degradations in the wild. Current solutions are often limited, as they either depend on specific degradation priors or suffer fr…
Autonomous DrivingImage RestorationVISIR: Visual and Semantic Image Label Refinement
The social media explosion has populated the Internet with a wealth of images. There are two existing paradigms for image retrieval: 1) content-based image retrieval (CBIR), which has traditionally used visual features f…
Content-Based Image RetrievalImage Retrievalobject-detectionObject Detection+23D-RETR: End-to-End Single and Multi-View 3D Reconstruction with Transformers
3D reconstruction aims to reconstruct 3D objects from 2D views. Previous works for 3D reconstruction mainly focus on feature matching between views or using CNNs as backbones. Recently, Transformers have been shown effec…
3D ReconstructionDecoderMulti-View 3D ReconstructionSnakeVoxFormer: Transformer-based Single Image\\Voxel Reconstruction with Run Length Encoding
Deep learning-based 3D object reconstruction has achieved unprecedented results. Among those, the transformer deep neural model showed outstanding performance in many applications of computer vision. We introduce SnakeVo…
3D Object ReconstructionData CompressionObject ReconstructionR3D-SWIN:Use Shifted Window Attention for Single-View 3D Reconstruction
Recently, vision transformers have performed well in various computer vision tasks, including voxel 3D reconstruction. However, the windows of the vision transformer are not multi-scale, and there is no connection betwee…
3D ReconstructionSingle-View 3D Reconstruction