Using mixup as regularization and tuning hyper-parameters for ResNets
While novel computer vision architectures are gaining traction, the impact of model architectures is often related to changes or exploring in training methods. Identity mapping-based architectures ResNets and DenseNets have promised path-breaking results in the image classification task and are go-to methods for even now if the data given is fairly limited. Considering the ease of training with limited resources this work revisits the ResNets and improves the ResNet50 \cite{resnets} by using mixup data-augmentation as regularization and tuning the hyper-parameters.
Code (1)
Tasks
Data Augmentationimage-classificationImage ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit
The cost of hyperparameter tuning in deep learning has been rising with model sizes, prompting practitioners to find new tuning methods using a proxy of smaller networks. One such proposal uses $\mu$P parameterized netwo…
Hyperparameter Tuning and Implicit Regularization in Minibatch SGD
This paper makes two contributions towards understanding how the hyperparameters of stochastic gradient descent affect the final training loss and test accuracy of neural networks. First, we argue that stochastic gradien…
k-Mixup Regularization for Deep Learning via Optimal Transport
Mixup is a popular regularization technique for training deep neural networks that improves generalization and increases robustness to certain distribution shifts. It perturbs input training data in the direction of othe…
Adversarial RobustnessDeep LearningHyperparameter OptimizationField theory for optimal signal propagation in ResNets
Residual networks have significantly better trainability and thus performance than feed-forward networks at large depth. Introducing skip connections facilitates signal propagation to deeper layers. In addition, previous…
SensitivityScalable Gradient-Based Tuning of Continuous Regularization Hyperparameters
Hyperparameter selection generally relies on running multiple full training trials, with selection based on validation set performance. We propose a gradient-based approach for locally adjusting hyperparameters during tr…
Hyperparameter Optimization