R3Net: Random Weights, Rectifier Linear Units and Robustness for Artificial Neural Network
We consider a neural network architecture with randomized features, a sign-splitter, followed by rectified linear units (ReLU). We prove that our architecture exhibits robustness to the input perturbation: the output feature of the neural network exhibits a Lipschitz continuity in terms of the input perturbation. We further show that the network output exhibits a discrimination ability that inputs that are not arbitrarily close generate output vectors which maintain distance between each other obeying a certain lower bound. This ensures that two different inputs remain discriminable while contracting the distance in the output feature space.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Continuously Differentiable Exponential Linear Units
Exponential Linear Units (ELUs) are a useful rectifier for constructing deep learning architectures, as they may speed up and otherwise improve learning by virtue of not have vanishing gradients and by having mean activa…
On the Compressive Power of Deep Rectifier Networks for High Resolution Representation of Class Boundaries
This paper provides a theoretical justification of the superior classification performance of deep rectifier networks over shallow rectifier networks from the geometrical perspective of piecewise linear (PWL) classifier …
General ClassificationDeep Online Convex Optimization with Gated Games
Methods from convex optimization are widely used as building blocks for deep learning algorithms. However, the reasons for their empirical success are unclear, since modern convolutional networks (convnets), incorporatin…
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Recti…
General Classificationimage-classificationImage ClassificationSwitched Control Applied to a Totem-Pole Bridgeless Rectifier for Power Factor Correction
The wide range of operation of bridgeless rectifiers requires a control technique that guarantee robustness. Linear Power Factor Correction (PFC) control techniques, although effective, cannot guarantee such robustness. …