Segformer++: Efficient Token-Merging Strategies for High-Resolution Semantic Segmentation
Utilizing transformer architectures for semantic segmentation of high-resolution images is hindered by the attention's quadratic computational complexity in the number of tokens. A solution to this challenge involves decreasing the number of tokens through token merging, which has exhibited remarkable enhancements in inference speed, training efficiency, and memory utilization for image classification tasks. In this paper, we explore various token merging strategies within the framework of the Segformer architecture and perform experiments on multiple semantic segmentation and human pose estimation datasets. Notably, without model re-training, we, for example, achieve an inference acceleration of 61% on the Cityscapes dataset while maintaining the mIoU performance. Consequently, this paper facilitates the deployment of transformer-based architectures on resource-constrained devices and in real-time applications.
Code (2)
Tasks
image-classificationImage ClassificationPose EstimationSegmentationSemantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DA-SegFormer: Damage-Aware Semantic Segmentation for Fine-Grained Disaster Assessment
Rapid and accurate damage assessment following natural disasters is critical for effective emergency response. However, identifying fine-grained damage levels (e.g., distinguishing minor from major roof damage) in UAV im…
Semantic SegmentationSemantic Segmentation of Unmanned Aerial Vehicle Remote Sensing Images using SegFormer
The escalating use of Unmanned Aerial Vehicles (UAVs) as remote sensing platforms has garnered considerable attention, proving invaluable for ground object recognition. While satellite remote sensing images face limitati…
Object RecognitionSegmentationSemantic SegmentationSegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
We present SegFormer, a simple, efficient yet powerful semantic segmentation framework which unifies Transformers with lightweight multilayer perception (MLP) decoders. SegFormer has two appealing features: 1) SegFormer …
2D Semantic SegmentationC++ codeCrack SegmentationDecoder+2Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers
Vision transformers have delivered tremendous success in representation learning. This is primarily due to effective token mixing through self attention. However, this scales quadratically with the number of pixels, whic…
Computational EfficiencyOperator learningRepresentation LearningEfficient Token Mixing for Transformers via Adaptive Fourier Neural Operators
Vision transformers have delivered tremendous success in representation learning. This is primarily due to effective token mixing through self attention. However, this scales quadratically with the number of pixels, whic…
Computational EfficiencyOperator learningRepresentation Learning