MaxUp: Lightweight Adversarial Training With Data Augmentation Improves Neural Network Training

2021-06-19 · CVPR 2021 1 · Chengyue Gong, Tongzheng Ren, Mao Ye, Qiang Liu

We propose MaxUp, an embarrassingly simple, highly effective technique for improving the generalization performance of machine learning models, especially deep neural networks. The idea is to generate a set of augmented data with some random perturbations or transforms, and minimize the maximum, or worst case loss over the augmented data. By doing so, we implicitly introduce a smoothness or robustness regularization against the random perturbations, and hence improve the generation performance. For example, in the case of Gaussian perturbation, MaxUp is asymptotically equivalent to using the gradient norm of the loss as a penalty to encourage smoothness. We test MaxUp on a range of tasks, including image classification, language modeling, and adversarial certification, on which MaxUp consistently outperforms the existing best baseline methods, without introducing substantial computational overhead. In particular, we improve ImageNet classification from the accuracy 85.5% without extra data to 85.8%.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Data Augmentation image-classification Image ClassificationLanguage ModelingLanguage Modelling

Methods 이 논문이 사용한 방법론

MaxUp MaxUp is an adversarial data augmentation technique for improving the generalization performance of machine learning models. The idea is to generate a set of augmented data…

MaxUp: A Simple Way to Improve Generalization of Neural Network Training

2020-02-20 · Chengyue Gong, Tongzheng Ren, Mao Ye, Qiang Liu

We propose \emph{MaxUp}, an embarrassingly simple, highly effective technique for improving the generalization performance of machine learning models, especially deep neural networks. The idea is to generate a set of aug…

Few-Shot Image Classification General Classification image-classification Image Classification+2

Improving Generalization in Meta-Learning via Meta-Gradient Augmentation

2023-06-14 · Ren Wang, Haoliang Sun, Qi Wei, Xiushan Nie 외

Meta-learning methods typically follow a two-loop framework, where each loop potentially suffers from notorious overfitting, hindering rapid adaptation and generalization to new tasks. Existing schemes solve it by enhanc…

Few-Shot Learning Memorization Meta-Learning Network Pruning

Training Generative Adversarial Network-Based Vocoder with Limited Data Using Augmentation-Conditional Discriminator

2024-03-25 · Takuhiro Kaneko, Hirokazu Kameoka, Kou Tanaka

A generative adversarial network (GAN)-based vocoder trained with an adversarial discriminator is commonly used for speech synthesis because of its fast, lightweight, and high-quality characteristics. However, this data-…

Data Augmentation Generative Adversarial Network Speech Synthesis

Abex-rat: Synergizing Abstractive Augmentation and Adversarial Training for Classification of Occupational Accident Reports

2025-09-02 · Jian Chen, Jiabao Dou arxiv

The automatic classification of occupational accident reports is pivotal for workplace safety analysis but is persistently hindered by severe class imbalance and data scarcity. In this paper, we propose ABEX-RAT, a resou…

Data Augmentation

Lightweight Safety Guardrails via Synthetic Data and RL-guided Adversarial Training

2025-07-11 · Aleksei Ilin, Gor Matevosyan, Xueying Ma, Vladimir Eremin 외

We introduce a lightweight yet highly effective safety guardrail framework for language models, demonstrating that small-scale language models can achieve, and even surpass, the performance of larger counterparts in cont…

Generative Adversarial Network Synthetic Data Generation