Do We Really Need Adaptive Global Spatial Attention for Traffic Forecasting?
Existing traffic forecasting models commonly focus on extracting spatial dependencies, particularly global spatial information, which characterizes the representations obtained through interactions between each node and all nodes across the traffic network. However, the underlying mechanism by which global information is modeled and extracted remains insufficiently investigated. Whether global information must be extracted by high-degree-of-freedom adaptive attention or can be captured by a simple global aggregation operator remains unclear. For this purpose, we design a controlled ablation framework that replaces only the spatial mixing module to test attention-based global interaction. Across six traffic benchmarks, standard spatial attention yields relative MAE changes of $-1.58\%$ to $+1.26\%$ compared with uniform full-range mixing, and we observe no consistent advantage for standard spatial attention, while uniform full-range mixing reduces node-scale spatial-mixing complexity from $O(N^2)$ to $O(N)$. We further propose a hypothesized model that decomposes spatial attention into a row-uniform global background and a non-uniform residual. The residual shows dataset-dependent effects. Overall, uniform full-range mixing provides a strong global spatial baseline, while the non-uniform attention residual is not consistently beneficial across datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Information ExtractionSimilar Papers 제목 키워드 기반
Keep It SimPool: Who Said Supervised Transformers Suffer from Attention Deficit?
Convolutional networks and vision transformers have different forms of pairwise interactions, pooling across layers and pooling at the end of the network. Does the latter really need to be different? As a by-product of p…
Image ClassificationImage RetrievalMetric LearningObject Discovery+3Do We Really Need Graph Neural Networks for Traffic Forecasting?
Spatio-temporal graph neural networks (STGNN) have become the most popular solution to traffic forecasting. While successful, they rely on the message passing scheme of GNNs to establish spatial dependencies between node…
All the attention you need: Global-local, spatial-channel attention for image retrieval
We address representation learning for large-scale instance-level image retrieval. Apart from backbone, training pipelines and loss functions, popular approaches have focused on different spatial pooling and attention me…
AllImage RetrievalRepresentation LearningRetrievalLightWeather: Harnessing Absolute Positional Encoding to Efficient and Scalable Global Weather Forecasting
Recently, Transformers have gained traction in weather forecasting for their capability to capture long-term spatial-temporal correlations. However, their complex architectures result in large parameter counts and extend…
Weather ForecastingCAT: Learning to Collaborate Channel and Spatial Attention from Multi-Information Fusion
Channel and spatial attention mechanism has proven to provide an evident performance boost of deep convolution neural networks (CNNs). Most existing methods focus on one or run them parallel (series), neglecting the coll…
image-classificationImage ClassificationInstance Segmentationobject-detection+2