Tradeoffs in Data Augmentation: An Empirical Study
Though data augmentation has become a standard component of deep neural network training, the underlying mechanism behind the effectiveness of these techniques remains poorly understood. In practice, augmentation policies are often chosen using heuristics of distribution shift or augmentation diversity. Inspired by these, we conduct an empirical study to quantify how data augmentation improves model generalization. We introduce two interpretable and easy-to-compute measures: Affinity and Diversity. We find that augmentation performance is predicted not by either of these alone but by jointly optimizing the two.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationDiversitySimilar Papers 제목 키워드 기반
Counterfactual Fairness in Text Classification through Robustness
In this paper, we study counterfactual fairness in text classification, which asks the question: How would the prediction change if the sensitive attribute referenced in the example were different? Toxicity classifiers d…
AttributeClassificationcounterfactualFairness+3Confounding Tradeoffs for Neural Network Quantization
Many neural network quantization techniques have been developed to decrease the computational and memory footprint of deep learning. However, these methods are evaluated subject to confounding tradeoffs that may affect i…
QuantizationHow to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval
Various techniques have been developed in recent years to improve dense retrieval (DR), such as unsupervised contrastive learning and pseudo-query generation. Existing DRs, however, often suffer from effectiveness tradeo…
Contrastive LearningData AugmentationPassage RetrievalRetrieval+1Kolmogorov Arnold Networks (KANs) for Imbalanced Data -- An Empirical Perspective
Kolmogorov Arnold Networks (KANs) are recent architectural advancement in neural computation that offer a mathematically grounded alternative to standard neural networks. This study presents an empirical evaluation of KA…
Computational EfficiencyData AugmentationLangProBe: a Language Programs Benchmark
Composing language models (LMs) into multi-step language programs and automatically optimizing their modular prompts is now a mainstream paradigm for building AI systems, but the tradeoffs in this space have only scarcel…