Hyperparameter Selection Methods for Fitted Q-Evaluation with Error Guarantee
We are concerned with the problem of hyperparameter selection for the fitted Q-evaluation (FQE). FQE is one of the state-of-the-art method for offline policy evaluation (OPE), which is essential to the reinforcement learning without environment simulators. However, like other OPE methods, FQE is not hyperparameter-free itself and that undermines the utility in real-life applications. We address this issue by proposing a framework of approximate hyperparameter selection (AHS) for FQE, which defines a notion of optimality (called selection criteria) in a quantitative and interpretable manner without hyperparameters. We then derive four AHS methods each of which has different characteristics such as distribution-mismatch tolerance and time complexity. We also confirm in experiments that the error bound given by the theory matches empirical observations.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Sparse Feature Selection Makes Batch Reinforcement Learning More Sample Efficient
This paper provides a statistical analysis of high-dimensional batch Reinforcement Learning (RL) using sparse linear function approximation. When there is a large number of candidate features, our result sheds light on t…
feature selectionModel Selectionreinforcement-learningReinforcement Learning+1pared: Model selection using multi-objective optimization
Motivation: Model selection is a ubiquitous challenge in statistics. For penalized models, model selection typically entails tuning hyperparameters to maximize a measure of fit or minimize out-of-sample prediction error.…
modelModel SelectionZero-shot Outlier Detection via Prior-data Fitted Networks: Model Selection Bygone!
Outlier detection (OD) has a vast literature as it finds numerous applications in environmental monitoring, cybersecurity, finance, and medicine to name a few. Being an inherently unsupervised task, model selection is a …
Model SelectionOutlier DetectionModel Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settings
Reinforcement learning (RL) can be used to learn treatment policies and aid decision making in healthcare. However, given the need for generalization over complex state/action spaces, the incorporation of function approx…
Computational EfficiencyDecision MakingModel SelectionOffline RL+3Span error bound for weighted SVM with applications in hyperparameter selection
Weighted SVM (or fuzzy SVM) is the most widely used SVM variant owning its effectiveness to the use of instance weights. Proper selection of the instance weights can lead to increased generalization performance. In this …