$k$-Mixup Regularization for Deep Learning via Optimal Transport
Mixup is a popular regularization technique for training deep neural networks that can improve generalization and increase adversarial robustness. It perturbs input training data in the direction of other randomly-chosen instances in the training set. To better leverage the structure of the data, we extend mixup to $k$-mixup by perturbing $k$-batches of training points in the direction of other $k$-batches using displacement interpolation, i.e. interpolation under the Wasserstein metric. We demonstrate theoretically and in simulations that $k$-mixup preserves cluster and manifold structures, and we extend theory studying the efficacy of standard mixup to the $k$-mixup case. Our empirical results show that training with $k$-mixup further improves generalization and robustness across several network architectures and benchmark datasets of differing modalities.
Code (0)
등록된 구현이 없습니다.
Tasks
Adversarial RobustnessDeep LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Optimal transport meets noisy label robust loss and MixUp regularization for domain adaptation
It is common in computer vision to be confronted with domain shift: images which have the same class but different acquisition conditions. In domain adaptation (DA), one wants to classify unlabeled target images using so…
Domain Adaptationk-Mixup Regularization for Deep Learning via Optimal Transport
Mixup is a popular regularization technique for training deep neural networks that improves generalization and increases robustness to certain distribution shifts. It perturbs input training data in the direction of othe…
Adversarial RobustnessDeep LearningHyperparameter OptimizationCMOT: Cross-modal Mixup via Optimal Transport for Speech Translation
End-to-end speech translation (ST) is the task of translating speech signals in the source language into text in the target language. As a cross-modal task, end-to-end ST is difficult to train with limited data. Existing…
Machine TranslationTranslationPuzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup
While deep neural networks achieve great performance on fitting the training distribution, the learned networks are prone to overfitting and are susceptible to adversarial attacks. In this regard, a number of mixup based…
Adversarial RobustnessImage ClassificationSemantic SegmentationFused Gromov-Wasserstein Graph Mixup for Graph-level Classifications
Graph data augmentation has shown superiority in enhancing generalizability and robustness of GNNs in graph-level classifications. However, existing methods primarily focus on the augmentation in the graph signal space a…
Data Augmentation