paper-with-me

홈 › Papers

Dimension-Free Decision Calibration for Nonlinear Loss Functions

2025-04-22 · Jingwu Tang, Jiayun Wu, Zhiwei Steven Wu, Jiahao Zhang

When model predictions inform downstream decision making, a natural question is under what conditions can the decision-makers simply respond to the predictions as if they were the true outcomes. Calibration suffices to guarantee that simple best-response to predictions is optimal. However, calibration for high-dimensional prediction outcome spaces requires exponential computational and statistical complexity. The recent relaxation known as decision calibration ensures the optimality of the simple best-response rule while requiring only polynomial sample complexity in the dimension of outcomes. However, known results on calibration and decision calibration crucially rely on linear loss functions for establishing best-response optimality. A natural approach to handle nonlinear losses is to map outcomes $y$ into a feature space $\phi(y)$ of dimension $m$, then approximate losses with linear functions of $\phi(y)$. Unfortunately, even simple classes of nonlinear functions can demand exponentially large or infinite feature dimensions $m$. A key open problem is whether it is possible to achieve decision calibration with sample complexity independent of~$m$. We begin with a negative result: even verifying decision calibration under standard deterministic best response inherently requires sample complexity polynomial in~$m$. Motivated by this lower bound, we investigate a smooth version of decision calibration in which decision-makers follow a smooth best-response. This smooth relaxation enables dimension-free decision calibration algorithms. We introduce algorithms that, given $\mathrm{poly}(|A|,1/\epsilon)$ samples and any initial predictor~$p$, can efficiently post-process it to satisfy decision calibration without worsening accuracy. Our algorithms apply broadly to function classes that can be well-approximated by bounded-norm functions in (possibly infinite-dimensional) separable RKHS.

📄 PDF Abstract BibTeX arXiv:2504.15615

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Task-Aware Calibration: Provably Optimal Decoding in LLMs

2026-05-11 · Tim Tomov, Dominik Fuchsgruber, Rajeev Verma, Stephan Günnemann arxiv

LLM decoding often relies on the model's predictive distribution to generate an output. Consequently, misalignment with respect to the true generating distribution leads to suboptimal decisions in practice. While a natur…

Blade: A Derivative-free Bayesian Inversion Method using Diffusion Priors

2025-10-13 · Hongkai Zheng, Austin Wang, Zihui Wu, Zhengyu Huang 외 arxiv

Derivative-free Bayesian inversion arises in science and engineering applications, particularly when forward model is costly or infeasible to differentiate through. Existing derivative-free methods collapse the posterior…

Reliable Decisions with Threshold Calibration

2021-12-01 · NeurIPS 2021 12 · Roshni Sahoo, Shengjia Zhao, Alyssa Chen, Stefano Ermon

Decision makers rely on probabilistic forecasts to predict the loss of different decision rules before deployment. When the forecasted probabilities match the true frequencies, predicted losses will be accurate. Although…

Scheduling

Calibration Error for Decision Making

2024-04-21 · Lunjia Hu, Yifan Wu

Calibration allows predictions to be reliably interpreted as probabilities by decision makers. We propose a decision-theoretic calibration error, the Calibration Decision Loss (CDL), defined as the maximum improvement in…

Decision Making

Smooth Calibration and Decision Making

2025-04-22 · Jason Hartline, Yifan Wu, Yunran Yang

Calibration requires predictor outputs to be consistent with their Bayesian posteriors. For machine learning predictors that do not distinguish between small perturbations, calibration errors are continuous in prediction…

Decision Making