Regularity and Stability Properties of Selective SSMs with Discontinuous Gating
Deep Selective State-Space Models (SSMs), characterized by input-dependent, time-varying parameters, offer significant expressive power but pose challenges for stability analysis, especially with discontinuous gating signals. In this paper, we investigate the stability and regularity properties of continuous-time selective SSMs through the lens of passivity and Input-to-State Stability (ISS). We establish that intrinsic energy dissipation guarantees exponential forgetting of past states. Crucially, we prove that the unforced system dynamics possess an underlying minimal quadratic energy function whose defining matrix exhibits robust $\text{AUC}_{\text{loc}}$ regularity, accommodating discontinuous gating. Furthermore, assuming a universal quadratic storage function ensures passivity across all inputs, we derive parametric LMI conditions and kernel constraints that limit gating mechanisms, formalizing "irreversible forgetting" of recurrent models. Finally, we provide sufficient conditions for global ISS, linking uniform local dissipativity to overall system robustness. Our findings offer a rigorous framework for understanding and designing stable and reliable deep selective SSMs.
Code (0)
등록된 구현이 없습니다.
Tasks
State Space ModelsSimilar Papers 제목 키워드 기반
Mathematical Formalism for Memory Compression in Selective State Space Models
State space models (SSMs) have emerged as a powerful framework for modelling long-range dependencies in sequence data. Unlike traditional recurrent neural networks (RNNs) and convolutional neural networks (CNNs), SSMs of…
State Space ModelsTime Series ForecastingThe Curious Case of In-Training Compression of State Space Models
State Space Models (SSMs), developed to tackle long sequence modeling tasks efficiently, offer both parallelizable training and fast inference. At their core are recurrent dynamical systems that maintain a hidden state, …
Computational EfficiencyOn the Expressiveness and Length Generalization of Selective State-Space Models on Regular Languages
Selective state-space models (SSMs) are an emerging alternative to the Transformer, offering the unique advantage of parallel training and sequential inference. Although these models have shown promising performance on a…
State Space ModelsStability of the scattering transform for deformations with minimal regularity
Within the mathematical analysis of deep convolutional neural networks, the wavelet scattering transform introduced by St\'ephane Mallat is a unique example of how the ideas of multiscale analysis can be combined with a …
Bayesian Optimality of In-Context Learning with Selective State Spaces
We propose Bayesian optimal sequential prediction as a new principle for understanding in-context learning (ICL). Unlike interpretations framing Transformers as performing implicit gradient descent, we formalize ICL as m…