Papers Off-policy evaluation
“Off-policy evaluation” 태그가 달린 논문 265편 · 필터 해제
Off-Policy Evaluation and Learning for the Future under Non-Stationarity
We study the novel problem of future off-policy evaluation (F-OPE) and learning (F-OPL) for estimating and optimizing the future value of policies in non-stationary environments, where distributions vary over time. In e-…
Off-policy evaluationA Principled Path to Fitted Distributional Evaluation
In reinforcement learning, distributional off-policy evaluation (OPE) focuses on estimating the return distribution of a target policy using offline data collected under a different policy. This work focuses on extending…
Atari GamesOff-policy evaluationreinforcement-learningReinforcement LearningSemi-gradient DICE for Offline Constrained Reinforcement Learning
Stationary Distribution Correction Estimation (DICE) addresses the mismatch between the stationary distribution induced by a policy and the target distribution required for reliable off-policy evaluation (OPE) and policy…
Offline RLOff-policy evaluationreinforcement-learningReinforcement LearningSTITCH-OPE: Trajectory Stitching with Guided Diffusion for Off-Policy Evaluation
Off-policy evaluation (OPE) estimates the performance of a target policy using offline data collected from a behavior policy, and is crucial in domains such as robotics or healthcare where direct interaction with the env…
D4RLDenoisingOff-policy evaluationOpenAI GymCharacterization of Efficient Influence Function for Off-Policy Evaluation Under Optimal Policies
Off-policy evaluation (OPE) provides a powerful framework for estimating the value of a counterfactual policy using observational data, without the need for additional experimentation. Despite recent progress in robust a…
counterfactualOff-policy evaluationStabilizing Temporal Difference Learning via Implicit Stochastic Recursion
Temporal difference (TD) learning is a foundational algorithm in reinforcement learning (RL). For nearly forty years, TD learning has served as a workhorse for applied RL as well as a building block for more complex and …
Computational EfficiencyOff-policy evaluationReinforcement Learning (RL)DOLCE: Decomposing Off-Policy Evaluation/Learning into Lagged and Current Effects
Off-policy evaluation (OPE) and off-policy learning (OPL) for contextual bandit policies leverage historical data to evaluate and optimize a target policy. Most existing OPE/OPL methods--based on importance weighting or …
ImputationOff-policy evaluationOff-Policy Evaluation for Sequential Persuasion Process with Unobserved Confounding
In this paper, we expand the Bayesian persuasion framework to account for unobserved confounding variables in sender-receiver interactions. While traditional models assume that belief updates follow Bayesian principles, …
Decision MakingOff-policy evaluationSequential Decision MakingEnhancing Offline Model-Based RL via Active Model Selection: A Bayesian Optimization Perspective
Offline model-based reinforcement learning (MBRL) serves as a competitive framework that can learn well-performing policies solely from pre-collected data with the help of learned dynamics models. To fully unleash the po…
Bayesian OptimizationmodelModel-based Reinforcement LearningModel Selection+2Off-Policy Evaluation for Recommendations with Missing-Not-At-Random Rewards
Unbiased recommender learning (URL) and off-policy evaluation/learning (OPE/L) techniques are effective in addressing the data bias caused by display position and logging policies, thereby consistently improving the perf…
Off-policy evaluationPositionModel Selection for Off-policy Evaluation: New Algorithms and Experimental Protocol
Holdout validation and hyperparameter tuning from data is a long-standing problem in offline reinforcement learning (RL). A standard framework is to use off-policy evaluation (OPE) methods to evaluate and select the poli…
Model SelectionOff-policy evaluationReinforcement Learning (RL)Trajectory World Models for Heterogeneous Environments
Heterogeneity in sensors and actuators across environments poses a significant challenge to building large-scale pre-trained world models on top of this low-dimensional sensor information. In this work, we explore pre-tr…
DiversityModel Predictive ControlOff-policy evaluationOff-policy Evaluation for Payments at Adyen
This paper demonstrates the successful application of Off-Policy Evaluation (OPE) to accelerate recommender system development and optimization at Adyen, a global leader in financial payment processing. Facing the limita…
BenchmarkingDecision MakingOff-policy evaluationRecommendation SystemsOff-Policy Evaluation and Counterfactual Methods in Dynamic Auction Environments
Counterfactual estimators are critical for learning and refining policies using logged data, a process known as Off-Policy Evaluation (OPE). OPE allows researchers to assess new policies without costly experiments, speed…
counterfactualDecision MakingOff-policy evaluationCANDOR: Counterfactual ANnotated DOubly Robust Off-Policy Evaluation
Off-policy evaluation (OPE) provides safety guarantees by estimating the performance of a policy before deployment. Recent work introduced IS+, an importance sampling (IS) estimator that uses expert-annotated counterfact…
counterfactualOff-policy evaluationTwo-way Deconfounder for Off-policy Evaluation in Causal Reinforcement Learning
This paper studies off-policy evaluation (OPE) in the presence of unmeasured confounders. Inspired by the two-way fixed effects regression model widely used in the panel data literature, we propose a two-way unmeasured c…
Off-policy evaluationConcept-driven Off Policy Evaluation
Evaluating off-policy decisions using batch data poses significant challenges due to limited sample sizes leading to high variance. To improve Off-Policy Evaluation (OPE), we must identify and address the sources of this…
Off-policy evaluationLogarithmic Neyman Regret for Adaptive Estimation of the Average Treatment Effect
Estimation of the Average Treatment Effect (ATE) is a core problem in causal inference with strong connections to Off-Policy Evaluation in Reinforcement Learning. This paper considers the problem of adaptively selecting …
Causal InferenceOff-policy evaluationOff-policy estimation with adaptively collected data: the power of online learning
We consider estimation of a linear functional of the treatment effect using adaptively collected data. This task finds a variety of applications including the off-policy evaluation (\textsf{OPE}) in contextual bandits, a…
Causal InferenceMulti-Armed BanditsOff-policy evaluationMinimum Empirical Divergence for Sub-Gaussian Linear Bandits
We propose a novel linear bandit algorithm called LinMED (Linear Minimum Empirical Divergence), which is a linear extension of the MED algorithm that was originally designed for multi-armed bandits. LinMED is a randomize…
Multi-Armed BanditsOff-policy evaluationThompson Sampling