DeMansia: Mamba Never Forgets Any Tokens
This paper examines the mathematical foundations of transformer architectures, highlighting their limitations particularly in handling long sequences. We explore prerequisite models such as Mamba, Vision Mamba (ViM), and LV-ViT that pave the way for our proposed architecture, DeMansia. DeMansia integrates state space models with token labeling techniques to enhance performance in image classification tasks, efficiently addressing the computational challenges posed by traditional transformers. The architecture, benchmark, and comparisons with contemporary models demonstrate DeMansia's effectiveness. The implementation of this paper is available on GitHub at https://github.com/catalpaaa/DeMansia
Code (1)
Tasks
image-classificationImage ClassificationMambaState Space ModelsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
ZigzagPointMamba: Spatial-Semantic Mamba for Point Cloud Understanding
State Space models (SSMs) such as PointMamba enable efficient feature extraction for point cloud self-supervised learning with linear complexity, outperforming Transformers in computational efficiency. However, existing …
Computational EfficiencyMambaSelf-Supervised LearningState Space ModelsFaster Vision Mamba is Rebuilt in Minutes via Merged Token Re-training
Vision Mamba (e.g., Vim) has successfully been integrated into computer vision, and token reduction has yielded promising outcomes in Vision Transformers (ViTs). However, token reduction performs less effectively on Visi…
MambaToken ReductionSpectral-Spatial Mamba for Hyperspectral Image Classification
Recently, deep learning models have achieved excellent performance in hyperspectral image (HSI) classification. Among the many deep models, Transformer has gradually attracted interest for its excellence in modeling the …
ClassificationComputational EfficiencyHyperspectral Image Classificationimage-classification+2LoRA Learns Less and Forgets Less
Low-Rank Adaptation (LoRA) is a widely-used parameter-efficient finetuning method for large language models. LoRA saves memory by training only low rank perturbations to selected weight matrices. In this work, we compare…
PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition
We present PlainMamba: a simple non-hierarchical state space model (SSM) designed for general visual recognition. The recent Mamba model has shown how SSMs can be highly competitive with other architectures on sequential…
Image ClassificationInstance SegmentationMambaobject-detection+2