Dropout as data augmentation
Dropout is typically interpreted as bagging a large number of models sharing parameters. We show that using dropout in a network can also be interpreted as a kind of data augmentation in the input space without domain knowledge. We present an approach to projecting the dropout noise within a network back into the input space, thereby generating augmented versions of the training data, and we show that training a deterministic network on the augmented samples yields similar results. Finally, we propose a new dropout noise scheme based on our observations and show that it improves dropout results without adding significant computational cost.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
The Penalty Imposed by Ablated Data Augmentation
There is a set of data augmentation techniques that ablate parts of the input at random. These include input dropout, cutout, and random erasing. We term these techniques ablated data augmentation. Though these technique…
Data AugmentationData augmentation instead of explicit regularization
Contrary to most machine learning models, modern deep artificial neural networks typically include multiple components that contribute to regularization. Despite the fact that some (explicit) regularization techniques, s…
Data AugmentationObject CategorizationDropout Training for SVMs with Data Augmentation
Dropout and other feature noising schemes have shown promising results in controlling over-fitting by artificially corrupting the training data. Though extensive theoretical and empirical studies have been performed for …
Data AugmentationRepresentation LearningTackling Occlusion in Siamese Tracking with Structured Dropouts
Occlusion is one of the most difficult challenges in object tracking to model. This is because unlike other challenges, where data augmentation can be of help, occlusion is hard to simulate as the occluding object can be…
Data AugmentationObject TrackingAdjusting for Dropout Variance in Batch Normalization and Weight Initialization
We show how to adjust for the variance introduced by dropout with corrections to weight initialization and Batch Normalization, yielding higher accuracy. Though dropout can preserve the expected input to a neuron between…
Data Augmentation