TrackFormer: Multi-Object Tracking with Transformers

2021-01-07 · CVPR 2022 1 · Tim Meinhardt, Alexander Kirillov, Laura Leal-Taixe, Christoph Feichtenhofer

The challenging task of multi-object tracking (MOT) requires simultaneous reasoning about track initialization, identity, and spatio-temporal trajectories. We formulate this task as a frame-to-frame set prediction problem and introduce TrackFormer, an end-to-end trainable MOT approach based on an encoder-decoder Transformer architecture. Our model achieves data association between frames via attention by evolving a set of track predictions through a video sequence. The Transformer decoder initializes new tracks from static object queries and autoregressively follows existing tracks in space and time with the conceptually new and identity preserving track queries. Both query types benefit from self- and encoder-decoder attention on global frame-level features, thereby omitting any additional graph optimization or modeling of motion and/or appearance. TrackFormer introduces a new tracking-by-attention paradigm and while simple in its design is able to achieve state-of-the-art performance on the task of multi-object tracking (MOT17 and MOT20) and segmentation (MOTS20). The code is available at https://github.com/timmeinhardt/trackformer .

📄 PDF Abstract BibTeX arXiv:2101.02702

Code (2)

timmeinhardt/trackformer 공식 구현 pytorch

acaelles97/devis pytorch

Tasks

Decoder Multi-Object TrackingObjectObject TrackingVideo Understanding

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained Object Detectors

2022-11-17 · CVPR 2023 1 · Yuang Zhang, Tiancai Wang, Xiangyu Zhang

In this paper, we propose MOTRv2, a simple yet effective pipeline to bootstrap end-to-end multi-object tracking with a pretrained object detector. Existing end-to-end methods, MOTR and TrackFormer are inferior to their t…

Multi-Object Tracking Multiple Object Tracking Multiple Object Tracking with Transformer Multiple People Tracking+2

UniTrack: Differentiable Graph Representation Learning for Multi-Object Tracking

2026-02-04 · Bishoy Galoaa, Xiangyu Bai, Utsav Nandi, Sai Siddhartha Vivek Dhir Rangoju 외 arxiv

We present UniTrack, a plug-and-play graph-theoretic loss function designed to significantly enhance multi-object tracking (MOT) performance by directly optimizing tracking-specific objectives through unified differentia…

Graph Representation Learning Multi-Object Tracking

SOTFormer: A Minimal Transformer for Unified Object Tracking and Trajectory Prediction

2025-11-14 · Zhongping Dong, Pengyang Yu, Shuangjian Li, Liming Chen 외 arxiv

Accurate single-object tracking and short-term motion forecasting remain challenging under occlusion, scale variation, and temporal drift, which disrupt the temporal coherence required for real-time perception. We introd…

Trajectory Prediction Motion Forecasting Object Detection Object Tracking

TrackCore-F: Deploying Transformer-Based Subatomic Particle Tracking on FPGAs

2025-09-30 · Arjan Blankestijn, Uraz Odyurt, Amirreza Yousefzadeh arxiv

The Transformer Machine Learning (ML) architecture has been gaining considerable momentum in recent years. In particular, computational High-Energy Physics tasks such as jet tagging and particle track reconstruction (tra…

Jet Tagging

MOTR: End-to-End Multiple-Object Tracking with Transformer

2021-05-07 · Fangao Zeng, Bin Dong, Yuang Zhang, Tiancai Wang 외

Temporal modeling of objects is a key challenge in multiple object tracking (MOT). Existing methods track by associating detections through motion-based and appearance-based similarity heuristics. The post-processing nat…

Multi-Object Tracking Multiple Object Tracking Multiple Object Tracking with Transformer Object Detection+1