Replacing K-infinity Function with Leaky ReLU in Barrier Function Design: A Union of Invariant Sets Approach for ReLU-Based Dynamical Systems
In this paper, a systematic framework is presented for determining piecewise affine PWA barrier functions and their corresponding invariant sets for dynamical systems identified via Rectified Linear Unit (ReLU) neural networks or their equivalent PWA representations. A common approach to determining the invariant set is to use Nagumo's condition, or to utilize the barrier function with a class K-infinity function. It may be challenging to find a suitable class K-infinity function in some cases. We propose leaky ReLU as an efficient substitute for the complex nonlinear K-infinity function in our formulation. Moreover, we propose the Union of Invariant Sets (UIS) method, which combines information from multiple invariant sets in order to compute the largest possible PWA invariant set. The proposed framework is validated through multiple examples, showcasing its potential to enhance the analysis of invariant sets in ReLU-based dynamical systems. Our code is available at: https://github.com/PouyaSamanipour/UIS.git.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SAU: Smooth activation function using convolution with approximate identities
Well-known activation functions like ReLU or Leaky ReLU are non-differentiable at the origin. Over the years, many smooth approximations of ReLU have been proposed using various smoothing techniques. We propose new smoot…
Convergence of Deep Neural Networks with General Activation Functions and Pooling
Deep neural networks, as a powerful system to represent high dimensional complex functions, play a key role in deep learning. Convergence of deep neural networks is a fundamental issue in building the mathematical founda…
SMU: smooth activation function for deep networks using smoothing maximum technique
Deep learning researchers have a keen interest in proposing two new novel activation functions which can boost network performance. A good choice of activation function can have significant consequences in improving netw…
Deep LearningConvergence of Deep ReLU Networks
We explore convergence of deep neural networks with the popular ReLU activation function, as the depth of the networks tends to infinity. To this end, we introduce the notion of activation domains and activation matrices…
image-classificationImage ClassificationParametric Leaky Tanh: A New Hybrid Activation Function for Deep Learning
Activation functions (AFs) are crucial components of deep neural networks (DNNs), having a significant impact on their performance. An activation function in a DNN is typically a smooth, nonlinear function that transform…