홈 › Papers

Robust Generalization against Corruptions via Worst-Case Sharp ness Minimization

Robust generalization aims to deal with the most challenging data distributions which are rarely presented in training set and contain severe noise corruptions. Common solutions such as distributionally robust optimization (DRO) focus on the worst-case empirical risk to ensure low training error on the uncommon noisy distributions. However, due to the over-parameterized model being optimized on scarce worst-case data, DRO fails to produce a smooth loss landscape, thus struggling on generalizing well to the test set. Therefore, instead of focusing on the worst-case risk minimization, we propose SharpDRO by penalizing the sharpness of the worst-case distribution, which measures the loss changes around the neighbor of learning parameters. Through worst-case sharpness minimization, the proposed method successfully produces a flat loss curve on the corrupted distributions, thus achieving robust generalization. Moreover, by considering whether the distribution annotation is available, we apply SharpDRO to two problem settings and design a worst-case selection process for robust generalization. Through simulating real-world noisy distributions using CIFAR10/100 and ImageNet30 datasets, we show that SharpDRO exhibits strong generalization ability against severe corruptions and exceeds well-known baseline methods with large performance gains. Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반