Distributionally Robust Multimodal Machine Learning
We consider the problem of distributionally robust multimodal machine learning. Existing approaches often rely on merging modalities on the feature level (early fusion) or heuristic uncertainty modeling, which downplays modality-aware effects and provide limited insights. We propose a novel distributionally robust optimization (DRO) framework that aims to study both the theoretical and practical insights of multimodal machine learning. We first justify this setup and show the significance of this problem through complexity analysis. We then establish both generalization upper bounds and minimax lower bounds which provide performance guarantees. These results are further extended in settings where we consider encoder-specific error propogations. Empirically, we demonstrate that our approach improves robustness in both simulation settings and real-world datasets. Together, these findings provide a principled foundation for employing multimodal machine learning models in high-stakes applications where uncertainty is unavoidable.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Distributionally-Robust Machine Learning Using Locally Differentially-Private Data
We consider machine learning, particularly regression, using locally-differentially private datasets. The Wasserstein distance is used to define an ambiguity set centered at the empirical distribution of the dataset corr…
BIG-bench Machine LearningregressionConfidence Regions in Wasserstein Distributionally Robust Estimation
Wasserstein distributionally robust optimization estimators are obtained as solutions of min-max problems in which the statistician selects a parameter minimizing the worst-case loss among all probability models within a…
Distributional Robustness Bounds Generalization Errors
Bayesian methods, distributionally robust optimization methods, and regularization methods are three pillars of trustworthy machine learning combating distributional uncertainty, e.g., the uncertainty of an empirical dis…
Distributionally Robust Model Predictive Control with Mixture of Gaussian Processes
Despite the success of Gaussian process based Model Predictive Control (MPC) in robotic control, its applicability scope is greatly hindered by multimodal disturbances that are prevalent in real-world settings. Here we p…
Gaussian ProcessesModel Predictive ControlUnregularized limit of stochastic gradient method for Wasserstein distributionally robust optimization
Distributionally robust optimization offers a compelling framework for model fitting in machine learning, as it systematically accounts for data uncertainty. Focusing on Wasserstein distributionally robust optimization, …