Implicit Rugosity Regularization via Data Augmentation
Deep (neural) networks have been applied productively in a wide range of supervised and unsupervised learning tasks. Unlike classical machine learning algorithms, deep networks typically operate in the \emph{overparameterized} regime, where the number of parameters is larger than the number of training data points. Consequently, understanding the generalization properties and the role of (explicit or implicit) regularization in these networks is of great importance. In this work, we explore how the oft-used heuristic of \emph{data augmentation} imposes an {\em implicit regularization} penalty of a novel measure of the \emph{rugosity} or "roughness" based on the tangent Hessian of the function fit to the training data.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationSimilar Papers 제목 키워드 기반
Data augmentation instead of explicit regularization
Contrary to most machine learning models, modern deep artificial neural networks typically include multiple components that contribute to regularization. Despite the fact that some (explicit) regularization techniques, s…
Data AugmentationObject CategorizationEICO: Improving Few-Shot Text Classification via Explicit and Implicit Consistency Regularization
While the prompt-based fine-tuning methods had advanced few-shot natural language understanding tasks, self-training methods are also being explored. This work revisits the consistency regularization in self-training and…
Few-Shot LearningFew-Shot Text ClassificationLanguage ModelingLanguage Modelling+4Further advantages of data augmentation on convolutional neural networks
Data augmentation is a popular technique largely used to enhance the training of convolutional neural networks. Although many of its benefits are well known by deep learning researchers and practitioners, its implicit re…
Data AugmentationAspect-Based Sentiment Analysis with Explicit Sentiment Augmentations
Aspect-based sentiment analysis (ABSA), a fine-grained sentiment classification task, has received much attention recently. Many works investigate sentiment information through opinion words, such as ''good'' and ''bad''…
Aspect-Based Sentiment AnalysisAspect-Based Sentiment Analysis (ABSA)SentenceSentiment Analysis+1The good, the bad and the ugly sides of data augmentation: An implicit spectral regularization perspective
Data augmentation (DA) is a powerful workhorse for bolstering performance in modern machine learning. Specific augmentations like translations and scaling in computer vision are traditionally believed to improve generali…
Data Augmentationregression