Rethinking Transformer-based Set Prediction for Object Detection
DETR is a recently proposed Transformer-based method which views object detection as a set prediction problem and achieves state-of-the-art performance but demands extra-long training time to converge. In this paper, we investigate the causes of the optimization difficulty in the training of DETR. Our examinations reveal several factors contributing to the slow convergence of DETR, primarily the issues with the Hungarian loss and the Transformer cross-attention mechanism. To overcome these issues we propose two solutions, namely, TSP-FCOS (Transformer-based Set Prediction with FCOS) and TSP-RCNN (Transformer-based Set Prediction with RCNN). Experimental results show that the proposed methods not only converge much faster than the original DETR, but also significantly outperform DETR and other baselines in terms of detection accuracy.
Code (1)
Tasks
Objectobject-detectionObject DetectionPredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detection
Can Transformer perform 2D object- and region-level recognition from a pure sequence-to-sequence perspective with minimal knowledge about the 2D spatial structure? To answer this question, we present You Only Look at One…
Objectobject-detectionObject DetectionShuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer
Very recently, Window-based Transformers, which computed self-attention within non-overlapping local windows, demonstrated promising results on image classification, semantic segmentation, and object detection. However, …
image-classificationImage Classificationobject-detectionObject Detection+2FreqPDE: Rethinking Positional Depth Embedding for Multi-View 3D Object Detection Transformers
Detecting 3D objects accurately from multi-view 2D images is a challenging yet essential task in the field of autonomous driving. Current methods resort to integrating depth prediction to recover the spatial information …
3D Object DetectionAutonomous DrivingEmbracing Single Stride 3D Object Detector with Sparse Transformer
In LiDAR-based 3D object detection for autonomous driving, the ratio of the object size to input scene size is significantly smaller compared to 2D detection cases. Overlooking this difference, many 3D detectors directly…
3D Object DetectionAutonomous DrivingObjectobject-detection+2GLACIER: Rethinking Mass Spectrum Prediction as an Object Detection Problem
Predicting tandem mass spectra (MS/MS) from molecular structures represents a central task in analytical chemistry with direct relevance to clinical metabolomics, systems biology, and adjacent disciplines. In this work, …
Object Detection