Concentration inequalities and optimal number of layers for stochastic deep neural networks
We state concentration inequalities for the output of the hidden layers of a stochastic deep neural network (SDNN), as well as for the output of the whole SDNN. These results allow us to introduce an expected classifier (EC), and to give probabilistic upper bound for the classification error of the EC. We also state the optimal number of layers for the SDNN via an optimal stopping procedure. We apply our analysis to a stochastic version of a feedforward neural network with ReLU activation function.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Concentration of dynamic risk measures in a Brownian filtration
Motivated by liquidity risk in mathematical finance, D. Lacker introduced concentration inequalities for risk measures, i.e. upper bounds on the \emph{liquidity risk profile} of a financial loss. We derive these inequali…
Concentration Inequalities for the Stochastic Optimization of Unbounded Objectives with Application to Denoising Score Matching
We derive novel concentration inequalities that bound the statistical error for a large class of stochastic optimization problems, focusing on the case of unbounded objective functions. Our derivations utilize the follow…
DenoisingStochastic OptimizationA Concentration-Based Approach for Optimizing the Estimation Performance in Stochastic Sensor Selection
In this work, we consider a sensor selection drawn at random by a sampling with replacement policy for a linear time-invariant dynamical system subject to process and measurement noise. We employ the Kalman filter to est…
Concentration for matrix martingales in continuous time and microscopic activity of social networks
This paper gives new concentration inequalities for the spectral norm of a wide class of matrix martingales in continuous time. These results extend previously established Freedman and Bernstein inequalities for series o…
Stochastic Separability of Embedding Manifolds
Neurobiological studies and representation learning have observed that representations of objects belonging to the same category in high-dimensional neural spaces exhibit low-dimensional object manifold characteristics, …
Representation Learning