Statistical Decision Theory Respecting Stochastic Dominance
The statistical decision theory pioneered by Wald (1950) has used state-dependent mean loss (risk) to measure the performance of statistical decision functions across potential samples. We think it evident that evaluation of performance should respect stochastic dominance, but we do not see a compelling reason to focus exclusively on mean loss. We think it instructive to also measure performance by other functionals that respect stochastic dominance, such as quantiles of the distribution of loss. This paper develops general principles and illustrative applications for statistical decision theory respecting stochastic dominance. We modify the Wald definition of admissibility to an analogous concept of stochastic dominance (SD) admissibility, which uses stochastic dominance rather than mean sampling performance to compare alternative decision rules. We study SD admissibility in two relatively simple classes of decision problems that arise in treatment choice. We reevaluate the relationship between the MLE, James-Stein, and James-Stein positive part estimators from the perspective of SD admissibility. We consider alternative criteria for choice among SD-admissible rules. We juxtapose traditional criteria based on risk, regret, or Bayes risk with analogous ones based on quantiles of state-dependent sampling distributions or the Bayes distribution of loss.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Multivariate Stochastic Dominance via Optimal Transport and Applications to Models Benchmarking
Stochastic dominance is an important concept in probability theory, econometrics and social choice theory for robustly modeling agents' preferences between random outcomes. While many works have been dedicated to the uni…
BenchmarkingEconometricsStatistical Comparisons of Classifiers by Generalized Stochastic Dominance
Although being a crucial question for the development of machine learning algorithms, there is still no consensus on how to compare classifiers over multiple data sets with respect to several criteria. Every comparison f…
Max- and min-stability under first-order stochastic dominance
Max-stability is the property that taking a maximum between two inputs results in a maximum between two outputs. We study max-stability with respect to first-order stochastic dominance, the most fundamental notion of sto…
Background risk and small-stakes risk aversion
We show that under plausible levels of background risk, no theory of choice under risk -- such as expected utility theory, prospect theory, or rank dependent utility -- can simultaneously satisfy the following three econ…
Statistical Multicriteria Evaluation of LLM-Generated Text
Assessing the quality of LLM-generated text remains a fundamental challenge in natural language processing. Current evaluation approaches often rely on isolated metrics or simplistic aggregations that fail to capture the…
BenchmarkingDiversity