Learning Distributionally Robust Models at Scale via Composite Optimization
To train machine learning models that are robust to distribution shifts in the data, distributionally robust optimization (DRO) has been proven very effective. However, the existing approaches to learning a distributionally robust model either require solving complex optimization problems such as semidefinite programming or a first-order method whose convergence scales linearly with the number of data samples -- which hinders their scalability to large datasets. In this paper, we show how different variants of DRO are simply instances of a finite-sum composite optimization for which we provide scalable methods. We also provide empirical results that demonstrate the effectiveness of our proposed algorithm with respect to the prior art in order to learn robust models from very large datasets.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Consistent Approximations in Composite Optimization
Approximations of optimization problems arise in computational procedures and sensitivity analysis. The resulting effect on solutions can be significant, with even small approximations of components of a problem translat…
Stochastic OptimizationDRO: A Python Library for Distributionally Robust Optimization in Machine Learning
We introduce dro, an open-source Python library for distributionally robust optimization (DRO) for regression and classification problems. The library implements 14 DRO formulations and 9 backbone models, enabling 79 dis…
Stochastic Gradient Methods for Distributionally Robust Optimization with f-divergences
We develop efficient solution methods for a robust empirical risk minimization problem designed to give calibrated confidence intervals on performance and provide optimal tradeoffs between bias and variance. Our methods …
Distributionally Robust Geometric Joint Chance-Constrained Optimization: Neurodynamic Approaches
This paper proposes a two-time scale neurodynamic duplex approach to solve distributionally robust geometric joint chance-constrained optimization problems. The probability distributions of the row vectors are not known …
Confidence Regions in Wasserstein Distributionally Robust Estimation
Wasserstein distributionally robust optimization estimators are obtained as solutions of min-max problems in which the statistician selects a parameter minimizing the worst-case loss among all probability models within a…