Contractive Rectifier Networks for Nonlinear Maximum Margin Classification
To find the optimal nonlinear separating boundary with maximum margin in the input data space, this paper proposes Contractive Rectifier Networks (CRNs), wherein the hidden-layer transformations are restricted to be contraction mappings. The contractive constraints ensure that the achieved separating margin in the input space is larger than or equal to the separating margin in the output layer. The training of the proposed CRNs is formulated as a linear support vector machine (SVM) in the output layer, combined with two or more contractive hidden layers. Effective algorithms have been proposed to address the optimization challenges arising from contraction constraints. Experimental results on MNIST, CIFAR-10, CIFAR-100 and MIT-67 datasets demonstrate that the proposed contractive rectifier networks consistently outperform their conventional unconstrained rectifier network counterparts.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationGeneral ClassificationSimilar Papers 제목 키워드 기반
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Recti…
General Classificationimage-classificationImage ClassificationOn the Compressive Power of Deep Rectifier Networks for High Resolution Representation of Class Boundaries
This paper provides a theoretical justification of the superior classification performance of deep rectifier networks over shallow rectifier networks from the geometrical perspective of piecewise linear (PWL) classifier …
General ClassificationFrom Deep to Shallow: Transformations of Deep Rectifier Networks
In this paper, we introduce transformations of deep rectifier networks, enabling the conversion of deep rectifier networks into shallow rectifier networks. We subsequently prove that any rectifier net of any depth can be…
A small-gain theorem for 2-contraction of nonlinear interconnected systems
This paper introduces small-gain sufficient conditions for $2$-contraction of feedback interconnected systems, on the basis of individual gains of suitable subsystems arising from a modular decomposition of the second ad…
Efficient Dual-Band Single-Port Rectifier for RF Energy Harvesting at FM and GSM Bands
This paper presents an efficient dual-band rectifier for radiofrequency energy harvesting (RFEH) applications at FM and GSM bands. The single-port rectifier circuit, which comprises a 3-port network, optimized T-matching…