Exploring vision transformer layer choosing for semantic segmentation

2023-05-02 · Fangjian Lin, Yizhe Ma, Shengwei Tian

Extensive work has demonstrated the effectiveness of Vision Transformers. The plain Vision Transformer tends to obtain multi-scale features by selecting fixed layers, or the last layer of features aiming to achieve higher performance in dense prediction tasks. However, this selection is often based on manual operation. And different samples often exhibit different features at different layers (e.g., edge, structure, texture, detail, etc.). This requires us to seek a dynamic adaptive fusion method to filter different layer features. In this paper, unlike previous encoder and decoder work, we design a neck network for adaptive fusion and feature selection, called ViTController. We validate the effectiveness of our method on different datasets and models and surpass previous state-of-the-art methods. Finally, our method can also be used as a plug-in module and inserted into different networks.

📄 PDF Abstract BibTeX arXiv:2305.01279

Code (0)

등록된 구현이 없습니다.

Tasks

Decoderfeature selectionSemantic Segmentation

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Adam 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Multi-Head Attention 설명 없음

Position-Wise Feed-Forward Layer 설명 없음

Convolutional Embedding Makes Hierarchical Vision Transformer Stronger

2022-07-27 · Cong Wang, Hongmin Xu, Xiong Zhang, Li Wang 외

Vision Transformers (ViTs) have recently dominated a range of computer vision tasks, yet it suffers from low training data efficiency and inferior local semantic representation capability without appropriate inductive bi…

Inductive Bias

Vision Transformer Finetuning Benefits from Non-Smooth Components

2026-02-06 · Ambroise Odonnat, Laetitia Chapel, Romain Tavenard, Ievgen Redko arxiv

The smoothness of the transformer architecture has been extensively studied in the context of generalization, training stability, and adversarial robustness. However, its role in transfer learning remains poorly understo…

Adversarial Robustness Transfer Learning

ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias

2021-06-07 · NeurIPS 2021 12 · Yufei Xu, Qiming Zhang, Jing Zhang, DaCheng Tao

Transformers have shown great potential in various computer vision tasks owing to their strong capability in modeling long-range dependency using the self-attention mechanism. Nevertheless, vision transformers treat an i…

Image Classification Inductive Bias Object Detection Video Object Segmentation

Other Tokens Matter: Exploring Global and Local Features of Vision Transformers for Object Re-Identification

2024-04-23 · Yingquan Wang, Pingping Zhang, Dong Wang, Huchuan Lu

Object Re-Identification (Re-ID) aims to identify and retrieve specific objects from images captured at different places and times. Recently, object Re-ID has achieved great success with the advances of Vision Transforme…

Object

Locating and Editing Figure-Ground Organization in Vision Transformers

2026-03-06 · Stefan Arnold, René Gröbner arxiv

Vision Transformers must resolve figure-ground organization by choosing between completions driven by local geometric evidence and those favored by global organizational priors, giving rise to a characteristic perceptual…