Survey of Dropout Methods for Deep Neural Networks
Dropout methods are a family of stochastic techniques used in neural network training or inference that have generated significant research interest and are widely used in practice. They have been successfully applied in neural network regularization, model compression, and in measuring the uncertainty of neural network outputs. While original formulated for dense neural network layers, recent advances have made dropout methods also applicable to convolutional and recurrent neural network layers. This paper summarizes the history of dropout methods, their various applications, and current areas of research interest. Important proposed methods are described in additional detail.
Code (0)
등록된 구현이 없습니다.
Tasks
Model CompressionSurveyMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Survey on Dropout Methods and Experimental Verification in Recommendation
Overfitting is a common problem in machine learning, which means the model too closely fits the training data while performing poorly in the test data. Among various methods of coping with overfitting, dropout is one of …
SurveyPrior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation
Popular approaches for quantifying predictive uncertainty in deep neural networks often involve distributions over weights or multiple models, for instance via Markov Chain sampling, ensembling, or Monte Carlo dropout. T…
SurveyDADEE: Well-calibrated uncertainty quantification in neural networks for barriers-based robot safety
Uncertainty-aware controllers that guarantee safety are critical for safety critical applications. Among such controllers, Control Barrier Functions (CBFs) based approaches are popular because they are fast, yet safe. Ho…
Gaussian ProcessesUncertainty QuantificationA Re-examination of Neural Selective Prediction for Natural Language Processing
We provide a survey and careful empirical comparison of the state-of-the-art in neural selective classification for NLP tasks. Across multiple trials on multiple datasets, only one of the surveyed techniques -- Monte Car…
PredictionA Survey on Evidential Deep Learning For Single-Pass Uncertainty Estimation
Popular approaches for quantifying predictive uncertainty in deep neural networks often involve a set of weights or models, for instance via ensembling or Monte Carlo Dropout. These techniques usually produce overhead by…
Survey