Attention Is not Everything: Efficient Alternatives for Vision
Recently computer vision has seen advancements mainly thanks to Transformer-based models. However many non-Transformer methods are still doing well being a direct competition of Transformer-based models. This review tries to present a comprehensive taxonomy of such methods and organize these methods into categories like convolution-based models, MLP-based models, state-space-based and more. These methods are looked at in terms of how efficient they are, how well they scale, how easy they are to understand and how robust they are. A total of 40 papers were chosen for this study. The goal is to give a view of non-Transformer methods and find out what challenges and opportunities exist for future computer vision research.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer
In this paper, we investigate the application of Vehicle-to-Everything (V2X) communication to improve the perception performance of autonomous vehicles. We present a robust cooperative perception framework with V2X commu…
3D Object DetectionAutonomous Vehiclesobject-detectionObject DetectionGrounding Everything: Emerging Localization Properties in Vision-Language Transformers
Vision-language foundation models have shown remarkable performance in various zero-shot settings such as image retrieval, classification, or captioning. But so far, those models seem to fall behind when it comes to zero…
Image RetrievalObject LocalizationSemantic SegmentationZero Shot SegmentationAdversarial Normalization: I Can Visualize Everything (ICE)
Vision transformers use [CLS] tokens to predict image classes. Their explainability visualization has been studied using relevant information from [CLS] tokens or focusing on attention scores during self-attention. S…
Object DiscoveryObject LocalizationWeakly-Supervised Object Localization6G Communication: Envisioning the Key Issues and Challenges
In 2030, we are going to evidence the 6G mobile communication technology, which will enable the Internet of Everything. Yet 5G has to be experienced by people worldwide and B5G has to be developed; the researchers have a…
Random Attention and Unobserved Reference Alternatives
In this paper, I develop and characterize a random attention model with unobserved reference alternatives. The decision-maker pays attention to different subsets of the available set of alternatives randomly. The referen…