UP-DETR: Unsupervised Pre-training for Object Detection with Transformers
DEtection TRansformer (DETR) for object detection reaches competitive performance compared with Faster R-CNN via a transformer encoder-decoder architecture. However, trained with scratch transformers, DETR needs large-scale training data and an extreme long training schedule even on COCO dataset. Inspired by the great success of pre-training transformers in natural language processing, we propose a novel pretext task named random query patch detection in Unsupervised Pre-training DETR (UP-DETR). Specifically, we randomly crop patches from the given image and then feed them as queries to the decoder. The model is pre-trained to detect these query patches from the input image. During the pre-training, we address two critical issues: multi-task learning and multi-query localization. (1) To trade off classification and localization preferences in the pretext task, we find that freezing the CNN backbone is the prerequisite for the success of pre-training transformers. (2) To perform multi-query localization, we develop UP-DETR with multi-query patch detection with attention mask. Besides, UP-DETR also provides a unified perspective for fine-tuning object detection and one-shot detection tasks. In our experiments, UP-DETR significantly boosts the performance of DETR with faster convergence and higher average precision on object detection, one-shot detection and panoptic segmentation. Code and pre-training models: https://github.com/dddzg/up-detr.
Code (2)
Tasks
DecoderMulti-Task Learningobject-detectionObject DetectionPanoptic SegmentationUnsupervised Pre-trainingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Object Detection with Transformers: A Review
The astounding performance of transformers in natural language processing (NLP) has motivated researchers to explore their applications in computer vision tasks. DEtection TRansformer (DETR) introduces transformers to ob…
2D Object DetectionObjectobject-detectionObject DetectionDETReg: Unsupervised Pretraining with Region Priors for Object Detection
Recent self-supervised pretraining methods for object detection largely focus on pretraining the backbone of the object detector, neglecting key parts of detection architecture. Instead, we introduce DETReg, a new self-s…
Few-Shot LearningFew-Shot Object DetectionObjectobject-detection+5An Empirical Study Of Self-supervised Learning Approaches For Object Detection With Transformers
Self-supervised learning (SSL) methods such as masked language modeling have shown massive performance gains by pretraining transformer models for a variety of natural language processing tasks. The follow-up research ad…
image-classificationImage ClassificationImage ReconstructionLanguage Modeling+6SeqCo-DETR: Sequence Consistency Training for Self-Supervised Object Detection with Transformers
Self-supervised pre-training and transformer-based networks have significantly improved the performance of object detection. However, most of the current self-supervised object detection methods are built on convolutiona…
Objectobject-detectionObject DetectionRepresentation Learningdetrex: Benchmarking Detection Transformers
The DEtection TRansformer (DETR) algorithm has received considerable attention in the research community and is gradually emerging as a mainstream approach for object detection and other perception tasks. However, the cu…
Benchmarkingobject-detectionObject DetectionPose Estimation