Universal generalization guarantees for Wasserstein distributionally robust models
Distributionally robust optimization has emerged as an attractive way to train robust machine learning models, capturing data uncertainty and distribution shifts. Recent statistical analyses have proved that generalization guarantees of robust models based on the Wasserstein distance have generalization guarantees that do not suffer from the curse of dimensionality. However, these results are either approximate, obtained in specific cases, or based on assumptions difficult to verify in practice. In contrast, we establish exact generalization guarantees that cover a wide range of cases, with arbitrary transport costs and parametric loss functions, including deep learning objectives with nonsmooth activations. We complete our analysis with an excess bound on the robust objective and an extension to Wasserstein robust models with entropic regularizations.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Exact Generalization Guarantees for (Regularized) Wasserstein Distributionally Robust Models
Wasserstein distributionally robust estimators have emerged as powerful models for prediction and decision-making under uncertainty. These estimators provide attractive generalization guarantees: the robust objective obt…
Decision MakingDecision Making Under UncertaintyGeneralizing to Unseen Domains with Wasserstein Distributional Robustness under Limited Source Knowledge
Domain generalization aims at learning a universal model that performs well on unseen target domains, incorporating knowledge from multiple source domains. In this research, we consider the scenario where different domai…
Domain GeneralizationRotated MNISTTest-time AdaptationFinite-Sample Guarantees for Wasserstein Distributionally Robust Optimization: Breaking the Curse of Dimensionality
Wasserstein distributionally robust optimization (DRO) aims to find robust and generalizable solutions by hedging against data perturbations in Wasserstein distance. Despite its recent empirical success in operations res…
Minimax Statistical Learning with Wasserstein Distances
As opposed to standard empirical risk minimization (ERM), distributionally robust optimization aims to minimize the worst-case risk over a larger ambiguity set containing the original empirical distribution of the traini…
Domain AdaptationGeneralization BoundsWasserstein Distributionally Robust Optimization: A Three-Player Game Framework
Wasserstein distributionally robust optimization (DRO) has recently received significant attention in machine learning due to its connection to generalization, robustness and regularization. Existing methods only conside…