Vision Transformer-Based Time-Series Image Reconstruction for Cloud-Filling Applications

2025-06-24 · Lujun Li, Yiqun Wang, Radu State

Cloud cover in multispectral imagery (MSI) poses significant challenges for early season crop mapping, as it leads to missing or corrupted spectral information. Synthetic aperture radar (SAR) data, which is not affected by cloud interference, offers a complementary solution, but lack sufficient spectral detail for precise crop mapping. To address this, we propose a novel framework, Time-series MSI Image Reconstruction using Vision Transformer (ViT), to reconstruct MSI data in cloud-covered regions by leveraging the temporal coherence of MSI and the complementary information from SAR from the attention mechanism. Comprehensive experiments, using rigorous reconstruction evaluation metrics, demonstrate that Time-series ViT framework significantly outperforms baselines that use non-time-series MSI and SAR or time-series MSI without SAR, effectively enhancing MSI image reconstruction in cloud-covered regions.

📄 PDF Abstract BibTeX arXiv:2506.19591

Code (0)

등록된 구현이 없습니다.

Tasks

Image ReconstructionTime Series

Methods 이 논문이 사용한 방법론

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Transformer A Transformer is a model architecture that eschews recurrence and instead relies entirely on an [attention…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Learning to Embed Time Series Patches Independently

2023-12-27 · Seunghan Lee, Taeyoung Park, Kibok Lee

Masked time series modeling has recently gained much attention as a self-supervised representation learning strategy for time series. Inspired by masked image modeling in computer vision, recent works first patchify and …

Contrastive Learning Representation Learning Time Series Time Series Forecasting

Temporal-Spatial Tubelet Embedding for Cloud-Robust MSI Reconstruction using MSI-SAR Fusion: A Multi-Head Self-Attention Video Vision Transformer Approach

2025-12-10 · Yiqun Wang, Lujun Li, Meiru Yue, Radu State arxiv

Cloud cover in multispectral imagery (MSI) significantly hinders early-season crop mapping by corrupting spectral information. Existing Vision Transformer(ViT)-based time-series reconstruction methods, like SMTS-ViT, oft…

Spectral Reconstruction

From Pixels to Predictions: Spectrogram and Vision Transformer for Better Time Series Forecasting

2024-03-17 · Zhen Zeng, Rachneet Kaur, Suchetha Siddagangappa, Tucker Balch 외

Time series forecasting plays a crucial role in decision-making across various domains, but it presents significant challenges. Recent studies have explored image-driven approaches using computer vision models to address…

Decision Making Time Series Time Series Forecasting

DDMT: Denoising Diffusion Mask Transformer Models for Multivariate Time Series Anomaly Detection

2023-10-13 · Chaocheng Yang, Tingyin Wang, Xuanhui Yan

Anomaly detection in multivariate time series has emerged as a crucial challenge in time series research, with significant research implications in various fields such as fraud detection, fault diagnosis, and system stat…

Anomaly Detection Denoising Fault Diagnosis Fraud Detection+3

VisionTS++: Cross-Modal Time Series Foundation Model with Continual Pre-trained Vision Backbones

2025-08-06 · Lefei Shen, Mouxiang Chen, Xu Liu, Han Fu 외 arxiv

Recent studies have indicated that vision models pre-trained on images can serve as time series foundation models (TSFMs) by reformulating time series forecasting (TSF) as image reconstruction. However, effective cross-m…

Time Series Forecasting Image Reconstruction