Pareto-Optimal Estimation and Policy Learning on Short-term and Long-term Treatment Effects

2024-03-05 · Yingrong Wang, Anpeng Wu, Haoxuan Li, Weiming Liu, Qiaowei Miao, Ruoxuan Xiong, Fei Wu, Kun Kuang

This paper focuses on developing Pareto-optimal estimation and policy learning to identify the most effective treatment that maximizes the total reward from both short-term and long-term effects, which might conflict with each other. For example, a higher dosage of medication might increase the speed of a patient's recovery (short-term) but could also result in severe long-term side effects. Although recent works have investigated the problems about short-term or long-term effects or the both, how to trade-off between them to achieve optimal treatment remains an open challenge. Moreover, when multiple objectives are directly estimated using conventional causal representation learning, the optimization directions among various tasks can conflict as well. In this paper, we systematically investigate these issues and introduce a Pareto-Efficient algorithm, comprising Pareto-Optimal Estimation (POE) and Pareto-Optimal Policy Learning (POPL), to tackle them. POE incorporates a continuous Pareto module with representation balancing, enhancing estimation efficiency across multiple tasks. As for POPL, it involves deriving short-term and long-term outcomes linked with various treatment levels, facilitating an exploration of the Pareto frontier emanating from these outcomes. Results on both the synthetic and real-world datasets demonstrate the superiority of our method.

📄 PDF Abstract BibTeX arXiv:2403.02624

Code (0)

등록된 구현이 없습니다.

Tasks

Representation Learning

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Deterministic Pareto-Optimal Policy Synthesis for Multi-Objective Reinforcement Learning

2026-06-24 · Aniruddha Joshi, Niklas Lauffer, Sanjit Seshia arxiv

Real-world decision-making often requires balancing multiple conflicting objectives, a challenge that standard Reinforcement Learning (RL) frequently addresses by aggregating rewards into a single scalar signal. While ef…

Reinforcement Learning

Pareto optimal proxy metrics

2023-07-03 · Alessandro Zito, Dylan Greaves, Jacopo Soriano, Lee Richardson

North star metrics and online experimentation play a central role in how technology companies improve their products. In many practical settings, however, evaluating experiments based on the north star metric directly ca…

Sensitivity

Policy Iteration for Pareto-Optimal Policies in Stochastic Stackelberg Games

2024-05-07 · Mikoto Kudo, Yohei Akimoto

In general-sum stochastic games, a stationary Stackelberg equilibrium (SSE) does not always exist, in which the leader maximizes leader's return for all the initial states when the follower takes the best response agains…

Pareto Policy Adaptation

2021-09-29 · ICLR 2022 4 · Panagiotis Kyriakis, Jyotirmoy Deshmukh, Paul Bogdan

We present a policy gradient method for Multi-Objective Reinforcement Learning under unknown, linear preferences. By enforcing Pareto stationarity, a first-order condition for Pareto optimality, we are able to design a s…

Multi-Objective Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Beyond ATE: Multi-Criteria Design for A/B Testing

2025-09-06 · Jiachun Li, Kaining Shi, David Simchi-Levi arxiv

In the era of large-scale AI deployment and high-stakes clinical trials, adaptive experimentation faces a ``trilemma'' of conflicting objectives: minimizing cumulative regret (welfare loss during the experiment), maximiz…