MaxDropoutV2: An Improved Method to Drop out Neurons in Convolutional Neural Networks
In the last decade, exponential data growth supplied the machine learning-based algorithms' capacity and enabled their usage in daily life activities. Additionally, such an improvement is partially explained due to the advent of deep learning techniques, i.e., stacks of simple architectures that end up in more complex models. Although both factors produce outstanding results, they also pose drawbacks regarding the learning process since training complex models denotes an expensive task and results are prone to overfit the training data. A supervised regularization technique called MaxDropout was recently proposed to tackle the latter, providing several improvements concerning traditional regularization approaches. In this paper, we present its improved version called MaxDropoutV2. Results considering two public datasets show that the model performs faster than the standard version and, in most cases, provides more accurate results.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Improved Dropout for Shallow and Deep Learning
Dropout has been witnessed with great success in training deep neural networks by independently zeroing out the outputs of neurons at random. It has also received a surge of interest for shallow learning, e.g., logistic …
Deep LearningStochastic OptimizationJumpout: Improved Dropout for Deep Neural Networks with Rectified Linear Units
Dropout is a simple yet effective technique to improve generalization performance and prevent overfitting in deep neural networks (DNNs). In this paper, we discuss three novel observations about dropout to better underst…
Adversarial Dropout for Recurrent Neural Networks
Successful application processing sequential data, such as text and speech, requires an improved generalization performance of recurrent neural networks (RNNs). Dropout techniques for RNNs were introduced to respond to t…
Language ModelingLanguage ModellingSemi-Supervised Text ClassificationText ClassificationMID-L: Matrix-Interpolated Dropout Layer with Layer-wise Neuron Selection
Modern neural networks often activate all neurons for every input, leading to unnecessary computation and inefficiency. We introduce Matrix-Interpolated Dropout Layer (MID-L), a novel module that dynamically selects and …
InformativenessA method of limiting performance loss of CNNs in noisy environments
Convolutional Neural Network (CNN) recognition rates drop in the presence of noise. We demonstrate a novel method of counteracting this drop in recognition rate by adjusting the biases of the neurons in the convolutional…
Denoising