Policy Learning with Asymmetric Counterfactual Utilities
Data-driven decision making plays an important role even in high stakes settings like medicine and public policy. Learning optimal policies from observed data requires a careful formulation of the utility function whose expected value is maximized across a population. Although researchers typically use utilities that depend on observed outcomes alone, in many settings the decision maker's utility function is more properly characterized by the joint set of potential outcomes under all actions. For example, the Hippocratic principle to "do no harm" implies that the cost of causing death to a patient who would otherwise survive without treatment is greater than the cost of forgoing life-saving treatment. We consider optimal policy learning with asymmetric counterfactual utility functions of this form that consider the joint set of potential outcomes. We show that asymmetric counterfactual utilities lead to an unidentifiable expected utility function, and so we first partially identify it. Drawing on statistical decision theory, we then derive minimax decision rules by minimizing the maximum expected utility loss relative to different alternative policies. We show that one can learn minimax loss decision rules from observed data by solving intermediate classification problems, and establish that the finite sample excess expected utility loss of this procedure is bounded by the regret of these intermediate classifiers. We apply this conceptual framework and methodology to the decision about whether or not to use right heart catheterization for patients with possible pulmonary hypertension.
Code (0)
등록된 구현이 없습니다.
Tasks
counterfactualDecision MakingSimilar Papers 제목 키워드 기반
Empirical Welfare Economics
Welfare economics relies on access to agents' utility functions: we revisit classical questions in welfare economics, assuming access to data on agents' past choices instead of their utilities. Our main result considers …
counterfactualCRAFT: Counterfactual-to-Interactive Reinforcement Fine-Tuning for Driving Policies
Open-loop imitation learning has advanced modern autonomous driving policy architectures, but closed-loop deployment remains vulnerable to policy-induced distribution shift. Existing post-training paradigms exhibit funda…
Autonomous DrivingPAC: Assisted Value Factorisation with Counterfactual Predictions in Multi-Agent Reinforcement Learning
Multi-agent reinforcement learning (MARL) has witnessed significant progress with the development of value function factorization methods. It allows optimizing a joint action-value function through the maximization of fa…
counterfactualMulti-agent Reinforcement Learningreinforcement-learningReinforcement Learning+4Policy Gradient for Reinforcement Learning with General Utilities
In Reinforcement Learning (RL), the goal of agents is to discover an optimal policy that maximizes the expected cumulative rewards. This objective may also be viewed as finding a policy that optimizes a linear function o…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Fairness Through Counterfactual Utilities
Group fairness definitions such as Demographic Parity and Equal Opportunity make assumptions about the underlying decision-problem that restrict them to classification problems. Prior work has translated these definition…
BIG-bench Machine LearningcounterfactualFairnessreinforcement-learning+2