Transformers in Vision: A Survey
Astounding results from Transformer models on natural language tasks have intrigued the vision community to study their application to computer vision problems. Among their salient benefits, Transformers enable modeling long dependencies between input sequence elements and support parallel processing of sequence as compared to recurrent networks e.g., Long short-term memory (LSTM). Different from convolutional networks, Transformers require minimal inductive biases for their design and are naturally suited as set-functions. Furthermore, the straightforward design of Transformers allows processing multiple modalities (e.g., images, videos, text and speech) using similar processing blocks and demonstrates excellent scalability to very large capacity networks and huge datasets. These strengths have led to exciting progress on a number of vision tasks using Transformer networks. This survey aims to provide a comprehensive overview of the Transformer models in the computer vision discipline. We start with an introduction to fundamental concepts behind the success of Transformers i.e., self-attention, large-scale pre-training, and bidirectional encoding. We then cover extensive applications of transformers in vision including popular recognition tasks (e.g., image classification, object detection, action recognition, and segmentation), generative modeling, multi-modal tasks (e.g., visual-question answering, visual reasoning, and visual grounding), video processing (e.g., activity recognition, video forecasting), low-level vision (e.g., image super-resolution, image enhancement, and colorization) and 3D analysis (e.g., point cloud classification and segmentation). We compare the respective advantages and limitations of popular techniques both in terms of architectural design and their experimental value. Finally, we provide an analysis on open research directions and possible future works.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionActivity RecognitionColorizationimage-classificationImage ClassificationImage EnhancementImage Super-Resolutionobject-detectionObject DetectionPoint Cloud ClassificationQuestion AnsweringSuper-ResolutionSurveyVideo ForecastingVisual GroundingVisual Question AnsweringVisual Question Answering (VQA)Visual ReasoningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A survey of the Vision Transformers and their CNN-Transformer based Variants
Vision transformers have become popular as a possible substitute to convolutional neural networks (CNNs) for a variety of computer vision applications. These transformers, with their ability to focus on global relationsh…
SurveyTransformers in Remote Sensing: A Survey
Deep learning-based algorithms have seen a massive popularity in different areas of remote sensing image analysis over the past decade. Recently, transformers-based architectures, originally introduced in natural languag…
SurveyA Survey of Vision Transformers in Autonomous Driving: Current Trends and Future Directions
This survey explores the adaptation of visual transformer models in Autonomous Driving, a transition inspired by their success in Natural Language Processing. Surpassing traditional Recurrent Neural Networks in tasks lik…
Autonomous DrivingDecoderLane Detectionobject-detection+43D Vision with Transformers: A Survey
The success of the transformer architecture in natural language processing has recently triggered attention in the computer vision field. The transformer has been used as a replacement for the widely used convolution ope…
Pose EstimationSurveyLarge Language Models Meet Computer Vision: A Brief Survey
Recently, the intersection of Large Language Models (LLMs) and Computer Vision (CV) has emerged as a pivotal area of research, driving significant advancements in the field of Artificial Intelligence (AI). As transformer…
Survey