paper-with-me

Papers

Spatiotemporal Transformer for Imputing Sparse Data: A Deep Learning Approach

2023-12-01 · Kehui Yao, Jingyi Huang, Jun Zhu

Effective management of environmental resources and agricultural sustainability heavily depends on accurate soil moisture data. However, datasets like the SMAP/Sentinel-1 soil moisture product often contain missing values across their spatiotemporal grid, which poses a significant challenge. This paper introduces a novel Spatiotemporal Transformer model (ST-Transformer) specifically designed to address the issue of missing values in sparse spatiotemporal datasets, particularly focusing on soil moisture data. The ST-Transformer employs multiple spatiotemporal attention layers to capture the complex spatiotemporal correlations in the data and can integrate additional spatiotemporal covariates during the imputation process, thereby enhancing its accuracy. The model is trained using a self-supervised approach, enabling it to autonomously predict missing values from observed data points. Our model's efficacy is demonstrated through its application to the SMAP 1km soil moisture data over a 36 x 36 km grid in Texas. It showcases superior accuracy compared to well-known imputation methods. Additionally, our simulation studies on other datasets highlight the model's broader applicability in various spatiotemporal imputation tasks.

📄 PDF Abstract BibTeX arXiv:2312.00963

Code (0)

등록된 구현이 없습니다.

Tasks

Deep LearningImputationManagementMissing Values

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Similar Papers 제목 키워드 기반

Causality-Aware Spatiotemporal Graph Neural Networks for Spatiotemporal Time Series Imputation

2024-03-18 · Baoyu Jing, Dawei Zhou, Kan Ren, Carl Yang

Spatiotemporal time series are usually collected via monitoring sensors placed at different locations, which usually contain missing values due to various failures, such as mechanical damages and Internet outages. Imputi…

Graph Neural NetworkImputationMissing ValuesTime Series

SSTVOS: Sparse Spatiotemporal Transformers for Video Object Segmentation

2021-01-21 · CVPR 2021 1 · Brendan Duke, Abdalla Ahmed, Christian Wolf, Parham Aarabi 외

In this paper we introduce a Transformer-based approach to video object segmentation (VOS). To address compounding error and scalability issues of prior work, we propose a scalable, end-to-end method for VOS called Spars…

Inductive BiasMotion SegmentationObjectOne-shot visual object segmentation+6

A Probabilistic Framework for Imputing Genetic Distances in Spatiotemporal Pathogen Models

2025-06-10 · Haley Stone, Jing Du, Hao Xue, Matthew Scotch 외

Pathogen genome data offers valuable structure for spatial models, but its utility is limited by incomplete sequencing coverage. We propose a probabilistic framework for inferring genetic distances between unsequenced ca…

Imputation

Table Transformers for Imputing Textual Attributes

2024-08-04 · Ting-Ruen Wei, YuAn Wang, Yoshitaka Inoue, Hsin-Tai Wu 외

Missing data in tabular dataset is a common issue as the performance of downstream tasks usually depends on the completeness of the training dataset. Previous missing data imputation methods focus on numeric and categori…

ImputationMulti-Task Learning

Kriformer: A Novel Spatiotemporal Kriging Approach Based on Graph Transformers

2024-09-23 · Renbin Pan, Feng Xiao, Hegui Zhang, Minyu Shen

Accurately estimating data in sensor-less areas is crucial for understanding system dynamics, such as traffic state estimation and environmental monitoring. This study addresses challenges posed by sparse sensor deployme…

Representation LearningState Estimation