Robust Convolution Neural ODEs via Contractivity-promoting regularization
Neural networks can be fragile to input noise and adversarial attacks. In this work, we consider Convolutional Neural Ordinary Differential Equations (NODEs), a family of continuous-depth neural networks represented by dynamical systems, and propose to use contraction theory to improve their robustness. For a contractive dynamical system two trajectories starting from different initial conditions converge to each other exponentially fast. Contractive Convolutional NODEs can enjoy increased robustness as slight perturbations of the features do not cause a significant change in the output. Contractivity can be induced during training by using a regularization term involving the Jacobian of the system dynamics. To reduce the computational burden, we show that it can also be promoted using carefully selected weight regularization terms for a class of NODEs with slope-restricted activation functions. The performance of the proposed regularizers is illustrated through benchmark image classification tasks on MNIST and FashionMNIST datasets, where images are corrupted by different kinds of noise and attacks.
Code (0)
등록된 구현이 없습니다.
Tasks
Image ClassificationSimilar Papers 제목 키워드 기반
Robust Classification using Contractive Hamiltonian Neural ODEs
Deep neural networks can be fragile and sensitive to small input perturbations that might cause a significant change in the output. In this paper, we employ contraction theory to improve the robustness of neural ODEs (NO…
Classificationimage-classificationImage ClassificationRobust classificationLocality-Promoting Representation Learning
This work investigates fundamental questions related to learning features in convolutional neural networks (CNN). Empirical findings across multiple architectures such as VGG, ResNet, Inception, DenseNet and MobileNet in…
Representation LearningFuzzy network jump models for soft dynamic clustering of graph-structured data
We introduce a fuzzy network jump model for clustering time-varying observations indexed by the nodes of a weighted graph. The framework allows flexible graph representations with spatial and temporal regularization prom…
Stochastic Logarithmic Lipschitz Constants: A Tool to Analyze Contractivity of Stochastic Differential Equations
We introduce the notion of stochastic logarithmic Lipschitz constants and use these constants to characterize stochastic contractivity of It\^o stochastic differential equations (SDEs) with multiplicative noise. We find …
Mitigating the Contractivity Trap in Diffusion ODEs via Stein Stabilization
A fundamental tension exists in the large-step inference of diffusion models via their deterministic probability flow ordinary differential equation (PF-ODE) trajectories, which we identify as the contractivity trap: eff…