Venturing into Uncharted Waters: The Navigation Compass from Transformer to Mamba
Transformer, a deep neural network architecture, has long dominated the field of natural language processing and beyond. Nevertheless, the recent introduction of Mamba challenges its supremacy, sparks considerable interest among researchers, and gives rise to a series of Mamba-based models that have exhibited notable potential. This survey paper orchestrates a comprehensive discussion, diving into essential research dimensions, covering: (i) the functioning of the Mamba mechanism and its foundation on the principles of structured state space models; (ii) the proposed improvements and the integration of Mamba with various networks, exploring its potential as a substitute for Transformers; (iii) the combination of Transformers and Mamba to compensate for each other's shortcomings. We have also made efforts to interpret Mamba and Transformer in the framework of kernel functions, allowing for a comparison of their mathematical nature within a unified context. Our paper encompasses the vast majority of improvements related to Mamba to date.
Code (0)
등록된 구현이 없습니다.
Tasks
MambaState Space ModelsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Navigating the Uncharted Waters: A Gradual Approach to the Certification and Integration of Maritime Autonomous Surface Ships (MASS) in Global Maritime Operations
The integration of Maritime Autonomous Surface Ships (MASS) into global maritime operations represents a transformative shift in the shipping industry, promising enhanced safety, efficiency, and cost-effectiveness. Howev…
Distribution-Calibrated Hierarchical Classification
While many advances have already been made on the topic of hierarchical classi- fication learning, we take a step back and examine how a hierarchical classifica- tion problem should be formally defined. We pay particular …
ClassificationGeneral ClassificationNavigating Data Heterogeneity in Federated Learning A Semi-Supervised Federated Object Detection
Federated Learning (FL) has emerged as a potent framework for training models across distributed data sources while maintaining data privacy. Nevertheless, it faces challenges with limited high-quality labels and non-IID…
Autonomous DrivingFederated LearningNavigateobject-detection+3Navigating Data Heterogeneity in Federated Learning: A Semi-Supervised Approach for Object Detection
Federated Learning (FL) has emerged as a potent framework for training models across distributed data sources while maintaining data privacy. Nevertheless, it faces challenges with limited high-quality labels and non-IID…
A Geometric Analysis-Based Safety Assessment Framework for MASS Route Decision-Making in Restricted Waters
To enhance the safety of Maritime Autonomous Surface Ships (MASS) navigating in restricted waters, this paper aims to develop a geometric analysis-based route safety assessment (GARSA) framework, specifically designed fo…
Decision Making