LyaNet: A Lyapunov Framework for Training Neural ODEs
We propose a method for training ordinary differential equations by using a control-theoretic Lyapunov condition for stability. Our approach, called LyaNet, is based on a novel Lyapunov loss formulation that encourages the inference dynamics to converge quickly to the correct prediction. Theoretically, we show that minimizing Lyapunov loss guarantees exponential convergence to the correct solution and enables a novel robustness guarantee. We also provide practical algorithms, including one that avoids the cost of backpropagating through a solver or using the adjoint method. Relative to standard Neural ODE training, we empirically find that LyaNet can offer improved prediction performance, faster convergence of inference dynamics, and improved adversarial robustness. Our code available at https://github.com/ivandariojr/LyapunovLearning .
Code (1)
Tasks
Adversarial RobustnessSimilar Papers 제목 키워드 기반
Tracking Finite-Time Lyapunov Exponents to Robustify Neural ODEs
We investigate finite-time Lyapunov exponents (FTLEs), a measure for exponential separation of input perturbations, of deep neural networks within the framework of continuous-depth neural ODEs. We demonstrate that FTLEs …
FxTS-Net: Fixed-Time Stable Learning Framework for Neural ODEs
Neural Ordinary Differential Equations (Neural ODEs), as a novel category of modeling big data methods, cleverly link traditional neural networks and dynamical systems. However, it is challenging to ensure the dynamics s…
A Lyapunov Condition for Training ODEs
Control theory is widely used in the study of differential equations to obtain desired behavior from underlying dynamics. We propose a novel method for training ordinary differential equations by using a control-theoret…
What ODE-Approximation Schemes of Time-Delay Systems Reveal about Lyapunov-Krasovskii Functionals
The article proposes an approach to complete-type and related Lyapunov-Krasovskii functionals that neither requires knowledge of the delay-Lyapunov matrix function nor does it involve linear matrix inequalities. The appr…
Numerical IntegrationLyapunov-Guided Training for Hardware-Safe Neural Networks Under Fixed-Point Arithmetic
Low-precision neural networks are attractive for resource-constrained hardware, but fixed-point arithmetic introduces failure modes that are often hidden by idealised quantisation models. In particular, two's-complement …