paper-with-me

홈 › Papers

Weaver: Kronecker Product Approximations of Spatiotemporal Attention for Traffic Network Forecasting

2025-11-12 · Christopher Cheong, Gary Davis, Seongjin Choi arxiv

Spatiotemporal forecasting on transportation networks is a complex task that requires understanding how traffic nodes interact within a dynamic, evolving system dictated by traffic flow dynamics and social behavioral patterns. The importance of transportation networks and ITS for modern mobility and commerce necessitates forecasting models that are not only accurate but also interpretable, efficient, and robust under structural or temporal perturbations. Recent approaches, particularly Transformer-based architectures, have improved predictive performance but often at the cost of high computational overhead and diminished architectural interpretability. In this work, we introduce Weaver, a novel attention-based model that applies Kronecker product approximations (KPA) to decompose the PN X PN spatiotemporal attention of O(P^2N^2) complexity into local P X P temporal and N X N spatial attention maps. This Kronecker attention map enables our Parallel-Kronecker Matrix-Vector product (P2-KMV) for efficient spatiotemporal message passing with O(P^2N + N^2P) complexity. To capture real-world traffic dynamics, we address the importance of negative edges in modeling traffic behavior by introducing Valence Attention using the continuous Tanimoto coefficient (CTC), which provides properties conducive to precise latent graph generation and training stability. To fully utilize the model's learning capacity, we introduce the Traffic Phase Dictionary for self-conditioning. Evaluations on PEMS-BAY and METR-LA show that Weaver achieves competitive performance across model categories while training more efficiently.

📄 PDF Abstract BibTeX arXiv:2511.08888

Code (0)

등록된 구현이 없습니다.

Tasks

Graph Generation

Similar Papers 제목 키워드 기반

Kronecker-Structured Nonparametric Spatiotemporal Point Processes

2026-03-24 · Zhitong Xu, Qiwei Yuan, Yinghao Chen, Yan Sun 외 arxiv

Events in spatiotemporal domains arise in numerous real-world applications, where uncovering event relationships and enabling accurate prediction are central challenges. Classical Poisson and Hawkes processes rely on res…

Point Processes

KoPA: Automated Kronecker Product Approximation

2019-12-05 · Chencheng Cai, Rong Chen, Han Xiao

We consider the problem of matrix approximation and denoising induced by the Kronecker product decomposition. Specifically, we propose to approximate a given matrix by the sum of a few Kronecker products of matrices, whi…

Denoising

Efficient Approximations of the Fisher Matrix in Neural Networks using Kronecker Product Singular Value Decomposition

2022-01-25 · Abdoulaye Koroko, Ani Anciaux-Sedrakian, Ibtihel Ben Gharbia, Valérie Garès 외

Several studies have shown the ability of natural gradient descent to minimize the objective function more efficiently than ordinary gradient descent based methods. However, the bottleneck of this approach for training d…

A Trace-restricted Kronecker-Factored Approximation to Natural Gradient

2020-11-21 · Kai-Xin Gao, Xiao-Lei Liu, Zheng-Hai Huang, Min Wang 외

Second-order optimization methods have the ability to accelerate convergence by modifying the gradient through the curvature matrix. There have been many attempts to use second-order optimization methods for training dee…

A Kronecker-factored approximate Fisher matrix for convolution layers

2016-02-03 · Roger Grosse, James Martens

Second-order optimization methods such as natural gradient descent have the potential to speed up training of neural networks by correcting for the curvature of the loss function. Unfortunately, the exact natural gradien…

Stochastic Optimization