Linear discriminant initialization for feed-forward neural networks
Informed by the basic geometry underlying feed forward neural networks, we initialize the weights of the first layer of a neural network using the linear discriminants which best distinguish individual classes. Networks initialized in this way take fewer training steps to reach the same level of training, and asymptotically have higher accuracy on training data.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
From Two-Class Linear Discriminant Analysis to Interpretable Multilayer Perceptron Design
A closed-form solution exists in two-class linear discriminant analysis (LDA), which discriminates two Gaussian-distributed classes in a multi-dimensional feature space. In this work, we interpret the multilayer perceptr…
Vocal Bursts Valence PredictionAn Effective and Efficient Initialization Scheme for Training Multi-layer Feedforward Neural Networks
Network initialization is the first and critical step for training neural networks. In this paper, we propose a novel network initialization scheme based on the celebrated Stein's identity. By viewing multi-layer feedfor…
The Shattered Gradients Problem: If resnets are the answer, then what is the question?
A long-standing obstacle to progress in deep learning is the problem of vanishing and exploding gradients. Although, the problem has largely been overcome via carefully constructed initializations and batch normalization…
Novel Architectures for Unsupervised Information Bottleneck based Speaker Diarization of Meetings
Speaker diarization is an important problem that is topical, and is especially useful as a preprocessor for conversational speech related applications. The objective of this paper is two-fold: (i) segment initialization …
Clusteringspeaker-diarizationSpeaker DiarizationLinear Discriminant Analysis with Gradient Optimization on Covariance Inverse
Linear discriminant analysis (LDA) is a fundamental method in statistical pattern recognition and classification, achieving Bayes optimality under Gaussian assumptions. However, it is well-known that classical LDA may st…