Unbiased Objective Estimation in Predictive Optimization
For data-driven decision-making, one promising approach, called predictive optimization, is to solve maximization problems i n which the objective function to be maximized is estimated from data. Predictive optimization, however, suffers from the problem of a calculated optimal solution’s being evaluated too optimistically, i.e., the value of the objective function is overestimated. This paper investigates such optimistic bias and presents two methods for correcting it. The first, which is analogous to cross-validation, successfully corrects the optimistic bias but results in underestimation of the true value. Our second method employs resampling techniques to avoid both overestimation and underestimation. We show that the second method, referred to as the parameter perturbation method, achieves asymptotically unbiased estimation. Empirical results for both artificial and real-world datasets demonstrate that our proposed approach successfully corrects the optimistic bias.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingSimilar Papers 제목 키워드 기반
A Stochastic Gradient Method with Biased Estimation for Faster Nonconvex Optimization
A number of optimization approaches have been proposed for optimizing nonconvex objectives (e.g. deep learning models), such as batch gradient descent, stochastic gradient descent and stochastic variance reduced gradient…
On Fenchel Mini-Max Learning
Inference, estimation, sampling and likelihood evaluation are four primary goals of probabilistic modeling. Practical considerations often force modeling approaches to make compromises between these objectives. We presen…
Density EstimationFine-Grained Dynamic Framework for Bias-Variance Joint Optimization on Data Missing Not at Random
In most practical applications such as recommendation systems, display advertising, and so forth, the collected data often contains missing values and those missing values are generally missing-not-at-random, which deter…
Generalization BoundsMissing ValuesRecommendation SystemsRigorous Evaluation of Predictive Toxicity Models by Multi-Objective Optimization of Reference Compound Lists Using Genetic Algorithms
In pharmaceutical safety assessments, validation studies are essential for evaluating the predictive performance and reliability of alternative methods prior to regulatory acceptance. Typically, these studies utilize ref…
DiversityOff-Policy Evaluation with Policy-Dependent Optimization Response
The intersection of causal inference and machine learning for decision-making is rapidly expanding, but the default decision criterion remains an \textit{average} of individual causal outcomes across a population. In pra…
Causal InferenceDecision MakingOff-policy evaluation