paper-with-me

홈 › Papers

Do We Really Need Adaptive Global Spatial Attention for Traffic Forecasting?

2026-07-14 · Qihang Zhang, Siyao Zhang, Letao Kang, Wenzhe Liang, Miao Zhang, Zhao Zhang arxiv

Existing traffic forecasting models commonly focus on extracting spatial dependencies, particularly global spatial information, which characterizes the representations obtained through interactions between each node and all nodes across the traffic network. However, the underlying mechanism by which global information is modeled and extracted remains insufficiently investigated. Whether global information must be extracted by high-degree-of-freedom adaptive attention or can be captured by a simple global aggregation operator remains unclear. For this purpose, we design a controlled ablation framework that replaces only the spatial mixing module to test attention-based global interaction. Across six traffic benchmarks, standard spatial attention yields relative MAE changes of $-1.58\%$ to $+1.26\%$ compared with uniform full-range mixing, and we observe no consistent advantage for standard spatial attention, while uniform full-range mixing reduces node-scale spatial-mixing complexity from $O(N^2)$ to $O(N)$. We further propose a hypothesized model that decomposes spatial attention into a row-uniform global background and a non-uniform residual. The residual shows dataset-dependent effects. Overall, uniform full-range mixing provides a strong global spatial baseline, while the non-uniform attention residual is not consistently beneficial across datasets.

📄 PDF Abstract BibTeX arXiv:2607.12462

Code (0)

등록된 구현이 없습니다.

Tasks

Information Extraction

Similar Papers 제목 키워드 기반

Keep It SimPool: Who Said Supervised Transformers Suffer from Attention Deficit?

2023-09-13 · ICCV 2023 1 · Bill Psomas, Ioannis Kakogeorgiou, Konstantinos Karantzalos, Yannis Avrithis

Convolutional networks and vision transformers have different forms of pairwise interactions, pooling across layers and pooling at the end of the network. Does the latter really need to be different? As a by-product of p…

Image ClassificationImage RetrievalMetric LearningObject Discovery+3

Do We Really Need Graph Neural Networks for Traffic Forecasting?

2023-01-30 · Xu Liu, Yuxuan Liang, Chao Huang, Hengchang Hu 외

Spatio-temporal graph neural networks (STGNN) have become the most popular solution to traffic forecasting. While successful, they rely on the message passing scheme of GNNs to establish spatial dependencies between node…

All the attention you need: Global-local, spatial-channel attention for image retrieval

2021-07-16 · Chull Hwan Song, Hye Joo Han, Yannis Avrithis

We address representation learning for large-scale instance-level image retrieval. Apart from backbone, training pipelines and loss functions, popular approaches have focused on different spatial pooling and attention me…

AllImage RetrievalRepresentation LearningRetrieval

LightWeather: Harnessing Absolute Positional Encoding to Efficient and Scalable Global Weather Forecasting

2024-08-19 · Yisong Fu, Fei Wang, Zezhi Shao, Chengqing Yu 외

Recently, Transformers have gained traction in weather forecasting for their capability to capture long-term spatial-temporal correlations. However, their complex architectures result in large parameter counts and extend…

Weather Forecasting

CAT: Learning to Collaborate Channel and Spatial Attention from Multi-Information Fusion

2022-12-13 · Zizhang Wu, Man Wang, Weiwei Sun, Yuchen Li 외

Channel and spatial attention mechanism has proven to provide an evident performance boost of deep convolution neural networks (CNNs). Most existing methods focus on one or run them parallel (series), neglecting the coll…

image-classificationImage ClassificationInstance Segmentationobject-detection+2