Transformer-Based Spatial-Temporal Counterfactual Outcomes Estimation

2025-06-26 · He Li, Haoang Chi, MingYu Liu, Wanrong Huang, Liyang Xu, Wenjing Yang

The real world naturally has dimensions of time and space. Therefore, estimating the counterfactual outcomes with spatial-temporal attributes is a crucial problem. However, previous methods are based on classical statistical models, which still have limitations in performance and generalization. This paper proposes a novel framework for estimating counterfactual outcomes with spatial-temporal attributes using the Transformer, exhibiting stronger estimation ability. Under mild assumptions, the proposed estimator within this framework is consistent and asymptotically normal. To validate the effectiveness of our approach, we conduct simulation experiments and real data experiments. Simulation experiments show that our estimator has a stronger estimation capability than baseline methods. Real data experiments provide a valuable conclusion to the causal effect of conflicts on forest loss in Colombia. The source code is available at https://github.com/lihe-maxsize/DeppSTCI_Release_Version-master.

📄 PDF Abstract BibTeX arXiv:2506.21154

Code (1)

lihe-maxsize/deppstci_release_version-master 공식 구현 pytorch

Tasks

counterfactual

Methods 이 논문이 사용한 방법론

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Transformer A Transformer is a model architecture that eschews recurrence and instead relies entirely on an [attention…

COSTAR: Improved Temporal Counterfactual Estimation with Self-Supervised Learning

2023-11-01 · Chuizheng Meng, Yihe Dong, Sercan Ö. Arik, Yan Liu 외

Estimation of temporal counterfactual outcomes from observed history is crucial for decision-making in many domains such as healthcare and e-commerce, particularly when randomized controlled trials (RCTs) suffer from hig…

counterfactual Decision Making Domain Adaptation Self-Supervised Learning+1

Longitudinal Targeted Minimum Loss-based Estimation with Temporal-Difference Heterogeneous Transformer

2024-04-05 · Toru Shirakawa, Yi Li, Yulun Wu, Sky Qiu 외

We propose Deep Longitudinal Targeted Minimum Loss-based Estimation (Deep LTMLE), a novel approach to estimate the counterfactual mean of outcome under dynamic treatment policies in longitudinal problem settings. Our app…

counterfactual Epidemiology Management

SCouT: Synthetic Counterfactuals via Spatiotemporal Transformers for Actionable Healthcare

2022-07-09 · Bhishma Dedhia, Roshini Balasubramanian, Niraj K. Jha

The Synthetic Control method has pioneered a class of powerful data-driven techniques to estimate the counterfactual reality of a unit from donor units. At its core, the technique involves a linear model fitted on the pr…

counterfactual Decision Making Decoder

Causal Transformer for Estimating Counterfactual Outcomes

2022-04-14 · Valentyn Melnychuk, Dennis Frauen, Stefan Feuerriegel

Estimating counterfactual outcomes over time from observational data is relevant for many applications (e.g., personalized medicine). Yet, state-of-the-art methods build upon simple long short-term memory (LSTM) networks…

counterfactual

Counterfactual Graph Transformer for Traffic Flow Prediction

2023-08-01 · Ying Yang, Kai Du, Xingyuan Dai, Jianwu Fang

Traffic flow prediction (TFP) is a fundamental problem of the Intelligent Transportation System (ITS), as it models the latent spatial-temporal dependency of traffic flow for potential congestion prediction. Recent graph…

counterfactual Prediction