A Generalization Bound for a Family of Implicit Networks
Implicit networks are a class of neural networks whose outputs are defined by the fixed point of a parameterized operator. They have enjoyed success in many applications including natural language processing, image processing, and numerous other applications. While they have found abundant empirical success, theoretical work on its generalization is still under-explored. In this work, we consider a large family of implicit networks defined parameterized contractive fixed point operators. We show a generalization bound for this class based on a covering number argument for the Rademacher complexity of these architectures.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Neural Optimization Kernel: Towards Robust Deep Learning
Deep neural networks (NN) have achieved great success in many applications. However, why do deep neural networks obtain good generalization at an over-parameterization regime is still unclear. To better understand deep N…
Deep LearningGeneralization BoundsSemi-Implicit Variational Inference
Semi-implicit variational inference (SIVI) is introduced to expand the commonly used analytic variational distribution family, by mixing the variational parameter with a flexible distribution. This mixing distribution ca…
Bayesian InferenceVariational InferenceOne Operator to Rule Them All? On Boundary-Indexed Operator Families in Neural PDE Solvers
Neural PDE solvers are often described as learning solution operators that map problem data to PDE solutions. In this work, we argue that this interpretation is generally incorrect when boundary conditions vary. We show …
Generalization bounds for neural ordinary differential equations and deep residual networks
Neural ordinary differential equations (neural ODEs) are a popular family of continuous-depth deep learning models. In this work, we consider a large family of parameterized ODEs with continuous-in-time parameters, whic…
Block-Sample MAC-Bayes Generalization Bounds
We present a family of novel block-sample MAC-Bayes bounds (mean approximately correct). While PAC-Bayes bounds (probably approximately correct) typically give bounds for the generalization error that hold with high prob…