Graph Signal Processing Meets Mamba2: Adaptive Filter Bank via Delta Modulation
State-space models (SSMs) offer efficient alternatives to attention with linear-time recurrence. Mamba2, a recent SSM-based language model, uses selective input gating and a multi-head structure, enabling parallel computation and strong benchmark performance. However, its multi-head recurrence operates independently without structured utilization or analysis. In this work, we propose a novel method called Hierarchical ADaptive filter bank for Efficient SSMs (HADES), a Graph Signal Processing (GSP)-inspired framework that reinterprets Mamba2 as an adaptive filter bank on a line graph. Our hierarchical architecture introduces two filter types: shared filters for global low-pass behavior and expert filters for local high-pass behavior, achieved through structured bias on the parameter Δ. HADES achieves comparable performance to baseline models including Mamba2 across various benchmarks in language modeling, commonsense reasoning, and long-context retrieval, while using only 58.9% of the original parameters. In this regard, HADES bridges GSP and neural sequence modeling, enabling efficient, hierarchical, and interpretable filtering within state-space models.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Mamba Meets Financial Markets: A Graph-Mamba Approach for Stock Price Prediction
Stock markets play an important role in the global economy, where accurate stock price predictions can lead to significant financial returns. While existing transformer-based models have outperformed long short-term memo…
MambaPredictionStock Price PredictionTime Series+1MSEMG: Surface Electromyography Denoising with a Mamba-based Efficient Network
Surface electromyography (sEMG) recordings can be contaminated by electrocardiogram (ECG) signals when the monitored muscle is closed to the heart. Traditional signal processing-based approaches, such as high-pass filter…
DenoisingMambaRhythmEEGMamba: Bidirectional State Space Model with Mixture of Experts for EEG Multi-task Classification
In recent years, with the development of deep learning, electroencephalogram (EEG) classification networks have achieved certain progress. Transformer-based models can perform well in capturing long-term dependencies in …
EEGElectroencephalogram (EEG)Emotion RecognitionMamba+5Adaptive State-Space Mamba for Real-Time Sensor Data Anomaly Detection
State-space modeling has emerged as a powerful paradigm for sequence analysis in various tasks such as natural language processing, time-series forecasting, and signal processing. In this work, we propose an \emph{Adapti…
Anomaly DetectionMambaState Space ModelsStyle Transfer+2Breaking Spatial Uniformity: Prior-Guided Mamba with Radial Serialization for Lens Flare Removal
Lens flares, caused by complex optical aberrations, severely degrade image quality especially in nighttime photography. Although recent restoration methods have made remarkable progress, most still rely on spatially unif…
Long-range modelingFlare Removal