Attention Mamba: Time Series Modeling with Adaptive Pooling Acceleration and Receptive Field Enhancements
"This work has been submitted to the lEEE for possible publication. Copyright may be transferred without noticeafter which this version may no longer be accessible." Time series modeling serves as the cornerstone of real-world applications, such as weather forecasting and transportation management. Recently, Mamba has become a promising model that combines near-linear computational complexity with high prediction accuracy in time series modeling, while facing challenges such as insufficient modeling of nonlinear dependencies in attention and restricted receptive fields caused by convolutions. To overcome these limitations, this paper introduces an innovative framework, Attention Mamba, featuring a novel Adaptive Pooling block that accelerates attention computation and incorporates global information, effectively overcoming the constraints of limited receptive fields. Furthermore, Attention Mamba integrates a bidirectional Mamba block, efficiently capturing long-short features and transforming inputs into the Value representations for attention mechanisms. Extensive experiments conducted on diverse datasets underscore the effectiveness of Attention Mamba in extracting nonlinear dependencies and enhancing receptive fields, establishing superior performance among leading counterparts. Our codes will be available on GitHub.
Code (0)
등록된 구현이 없습니다.
Tasks
MambaTime SeriesWeather ForecastingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Mamba or Transformer for Time Series Forecasting? Mixture of Universals (MoU) Is All You Need
Time series forecasting requires balancing short-term and long-term dependencies for accurate predictions. Existing methods mainly focus on long-term dependency modeling, neglecting the complexities of short-term dynamic…
AllMambaTime SeriesTime Series ForecastingMAAT: Mamba Adaptive Anomaly Transformer with association discrepancy for time series
Anomaly detection in time series is essential for industrial monitoring and environmental sensing, yet distinguishing anomalies from complex patterns remains challenging. Existing methods like the Anomaly Transformer and…
Anomaly DetectionAnomaly LocalizationMambaTime Series+1AdaMamba: Adaptive Frequency-Gated Mamba for Long-Term Time Series Forecasting
Accurate long-term time series forecasting (LTSF) requires the capture of complex long-range dependencies and dynamic periodic patterns. Recent advances in frequency-domain analysis offer a global perspective for uncover…
Computational EfficiencyTime Series ForecastingUniMamba: A Unified Spatial-Temporal Modeling Framework with State-Space and Attention Integration
Multivariate time series forecasting is fundamental to numerous domains such as energy, finance, and environmental monitoring, where complex temporal dependencies and cross-variable interactions pose enduring challenges.…
Multivariate Time Series ForecastingComputational EfficiencyDeMa: Dual-Path Delay-Aware Mamba for Efficient Multivariate Time Series Analysis
Accurate and efficient multivariate time series (MTS) analysis is increasingly critical for a wide range of intelligent applications. Within this realm, Transformers have emerged as the predominant architecture due to th…
Computational EfficiencyTime Series AnalysisAnomaly Detection