Boosted multilayer feedforward neural network with multiple output layers
This research introduces the Boosted Ensemble deep Multi-Layer Layer Perceptron (EdMLP) architecture with multiple output layers, a novel enhancement for the traditional Multi-Layer Perceptron (MLP). By adopting a layer-wise training approach, EdMLP enables the integration of boosting techniques within a single model, treating each layer as a weak learner, resulting in substantial performance gains. Additionally, the inclusion of layer-wise hyperparameter tuning allows optimization of individual layers thereby reducing the tuning time. Furthermore, the ensemble deep architecture’s versatility can be extended to other neural network-based models, such as the Self Normalized Network (SNN) where experiments demonstrate substantial performance enhancements yielded by the EdSNN compared to the standard original SNN model. This research underscores the potential of the EdMLP, and the Ed architecture in general as a powerful tool for improving the performance of various multilayer feedforward neural network models. The source code of this work is publicly accessible from the authors GitHub.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
From Two-Class Linear Discriminant Analysis to Interpretable Multilayer Perceptron Design
A closed-form solution exists in two-class linear discriminant analysis (LDA), which discriminates two Gaussian-distributed classes in a multi-dimensional feature space. In this work, we interpret the multilayer perceptr…
Vocal Bursts Valence PredictionImproving Transformer Models by Reordering their Sublayers
Multilayer transformer networks consist of interleaved self-attention and feedforward sublayers. Could ordering the sublayers in a different pattern lead to better performance? We generate randomly ordered transformers a…
Language ModelingLanguage ModellingMachine TranslationTranslationUniversal Approximation Theorem for Input-Connected Multilayer Perceptrons
We present the Input-Connected Multilayer Perceptron (IC-MLP), a feedforward neural network architecture in which each hidden neuron receives, in addition to the outputs of the preceding layer, a direct affine connection…
Competitive Learning with Feedforward Supervisory Signal for Pre-trained Multilayered Networks
We propose a novel learning method for multilayered neural networks which uses feedforward supervisory signal and associates classification of a new input with that of pre-trained input. The proposed method effectively u…
General ClassificationTheoretical Exploration of Solutions of Feedforward ReLU Networks
This paper aims to interpret the mechanism of feedforward ReLU networks by exploring their solutions for piecewise linear functions, through the deduction from basic rules. The constructed solution should be universal en…