paper-with-me

홈 › Papers

S2TNet: Spatio-Temporal Transformer Networks for Trajectory Prediction in Autonomous Driving

2022-06-22 · Weihuang Chen, Fangfang Wang, Hongbin Sun

To safely and rationally participate in dense and heterogeneous traffic, autonomous vehicles require to sufficiently analyze the motion patterns of surrounding traffic-agents and accurately predict their future trajectories. This is challenging because the trajectories of traffic-agents are not only influenced by the traffic-agents themselves but also by spatial interaction with each other. Previous methods usually rely on the sequential step-by-step processing of Long Short-Term Memory networks (LSTMs) and merely extract the interactions between spatial neighbors for single type traffic-agents. We propose the Spatio-Temporal Transformer Networks (S2TNet), which models the spatio-temporal interactions by spatio-temporal Transformer and deals with the temporel sequences by temporal Transformer. We input additional category, shape and heading information into our networks to handle the heterogeneity of traffic-agents. The proposed methods outperforms state-of-the-art methods on ApolloScape Trajectory dataset by more than 7\% on both the weighted sum of Average and Final Displacement Error. Our code is available at https://github.com/chenghuang66/s2tnet.

📄 PDF Abstract BibTeX arXiv:2206.10902

Code (1)

chenghuang66/s2tnet 공식 구현 pytorch

Tasks

Autonomous DrivingAutonomous VehiclesTrajectory Prediction

Methods 이 논문이 사용한 방법론

Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Position-Wise Feed-Forward Layer 설명 없음
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
Multi-Head Attention 설명 없음
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Similar Papers 제목 키워드 기반

Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction

2020-05-18 · ECCV 2020 8 · Cunjun Yu, Xiao Ma, Jiawei Ren, Haiyu Zhao 외

Understanding crowd motion dynamics is critical to real-world applications, e.g., surveillance systems and autonomous driving. This is challenging because it requires effectively modeling the socially aware crowd spatial…

Autonomous DrivingPedestrian Trajectory PredictionPredictionTrajectory Prediction

PastNet: Introducing Physical Inductive Biases for Spatio-temporal Video Prediction

2023-05-19 · Hao Wu, Fan Xu, Chong Chen, Xian-Sheng Hua 외

In this paper, we investigate the challenge of spatio-temporal video prediction task, which involves generating future video frames based on historical spatio-temporal observation streams. Existing approaches typically u…

PredictionVideo Prediction

MSSTNet: A Multi-Scale Spatio-Temporal CNN-Transformer Network for Dynamic Facial Expression Recognition

2024-04-12 · Linhuang Wang, Xin Kang, Fei Ding, Satoshi Nakagawa 외

Unlike typical video action recognition, Dynamic Facial Expression Recognition (DFER) does not involve distinct moving targets but relies on localized changes in facial muscles. Addressing this distinctive attribute, we …

Action RecognitionAttributeDynamic Facial Expression RecognitionFacial Expression Recognition+1

Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring

2024-06-11 · CVPR 2024 1 · Huicong Zhang, Haozhe Xie, Hongxun Yao

Video deblurring relies on leveraging information from other frames in the video sequence to restore the blurred regions in the current frame. Mainstream approaches employ bidirectional feature propagation, spatio-tempor…

DeblurringOptical Flow EstimationVideo Deblurring

TrajGPT: Controlled Synthetic Trajectory Generation Using a Multitask Transformer-Based Spatiotemporal Model

2024-11-07 · Shang-Ling Hsu, Emmanuel Tung, John Krumm, Cyrus Shahabi 외

Human mobility modeling from GPS-trajectories and synthetic trajectory generation are crucial for various applications, such as urban planning, disaster management and epidemiology. Both of these tasks often require fill…

EpidemiologyText Infilling