Improved Initialization of State-Space Artificial Neural Networks
The identification of black-box nonlinear state-space models requires a flexible representation of the state and output equation. Artificial neural networks have proven to provide such a representation. However, as in many identification problems, a nonlinear optimization problem needs to be solved to obtain the model parameters (layer weights and biases). A well-thought initialization of these model parameters can often avoid that the nonlinear optimization algorithm converges to a poorly performing local minimum of the considered cost function. This paper introduces an improved initialization approach for nonlinear state-space models represented as a recurrent artificial neural network and emphasizes the importance of including an explicit linear term in the model structure. Some of the neural network weights are initialized starting from a linear approximation of the nonlinear system, while others are initialized using random values or zeros. The effectiveness of the proposed initialization approach over previously proposed methods is illustrated on two benchmark examples.
Code (0)
등록된 구현이 없습니다.
Tasks
State Space ModelsSimilar Papers 제목 키워드 기반
Shrinkage Initialization for Smooth Learning of Neural Networks
The successes of intelligent systems have quite relied on the artificial learning of information, which lead to the broad applications of neural learning solutions. As a common sense, the training of neural networks can …
Common Sense ReasoningInitialization Approach for Nonlinear State-Space Identification via the Subspace Encoder Approach
The SUBNET neural network architecture has been developed to identify nonlinear state-space models from input-output data. To achieve this, it combines the rolled-out nonlinear state-space equations and a state encoder f…
State Space ModelsImproved weight initialization for deep and narrow feedforward neural network
Appropriate weight initialization settings, along with the ReLU activation function, have become cornerstones of modern deep learning, enabling the training and deployment of highly effective and efficient neural network…
Efficient Neural NetworkRandom Bias Initialization Improving Binary Neural Network Training
Edge intelligence especially binary neural network (BNN) has attracted considerable attention of the artificial intelligence community recently. BNNs significantly reduce the computational cost, model size, and memory fo…
Robustifying State-space Models for Long Sequences via Approximate Diagonalization
State-space models (SSMs) have recently emerged as a framework for learning long-range sequence tasks. An example is the structured state-space sequence (S4) layer, which uses the diagonal-plus-low-rank structure of the …
Computational EfficiencyState Space Models