Dither is Better than Dropout for Regularising Deep Neural Networks
Regularisation of deep neural networks (DNN) during training is critical to performance. By far the most popular method is known as dropout. Here, cast through the prism of signal processing theory, we compare and contrast the regularisation effects of dropout with those of dither. We illustrate some serious inherent limitations of dropout and demonstrate that dither provides a more effective regulariser.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Parallel Dither and Dropout for Regularising Deep Neural Networks
Effective regularisation during training can mean the difference between success and failure for deep neural networks. Recently, dither has been suggested as alternative to dropout for regularisation during batch-average…
Regularising Deep Networks with Deep Generative Models
We develop a new method for regularising neural networks. We learn a probability distribution over the activations of all layers of the model and then insert imputed values into the network during training. We obtain a p…
Data AugmentationImputationGIFnets: Differentiable GIF Encoding Framework
Graphics Interchange Format (GIF) is a widely used image file format. Due to the limited number of palette colors, GIF encoding often introduces color banding artifacts. Traditionally, dithering is applied to reduce colo…
Quantity over Quality: Dithered Quantization for Compressive Radar Systems
In this paper, we investigate a trade-off between the number of radar observations (or measurements) and their resolution in the context of radar range estimation. To this end, we introduce a novel estimation scheme that…
Compressive SensingQuantizationDistortion-Controlled Dithering with Reduced Recompression Rate
Dithering is a technique that can improve human perception of low-resolution data by reducing quantization artifacts. In this work we formalize and analytically justify two metrics for quantization artifact prominence, u…
Data CompressionImage CompressionQuantization