Using Common Random Numbers for Simulation-based Planning with Rollouts
Simulation-based planning with rollouts is a widely-deployed technique for decision making in stochastic environments. The primary instrument of simulation-based planning is a sampling model, which is repeatedly called to generate trajectories and estimate the utilities of available actions. Among the actions thus explored, one with the maximum estimated utility is then executed. In this paper, we examine the effect of using common random numbers in the simulation process. We obtain a simple recipe for (provably) reducing variance in relative utility when simulations invoke a rollout policy beyond some depth. Experiments on synthetic tasks confirm that our scheme improves task performance. The broader significance of our innovation is apparent from two practical applications: (1) single-step lookahead planning in a pension-disbursement task, and (2) a deployment of the well-known UCT algorithm for the game of Ludo.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingSimilar Papers 제목 키워드 기반
Bayesian Optimization Allowing for Common Random Numbers
Bayesian optimization is a powerful tool for expensive stochastic black-box optimization problems such as simulation-based optimization or machine learning hyperparameter tuning. Many stochastic objective functions impli…
Bayesian OptimizationBIG-bench Machine LearningNoise-free comparison of stochastic agent-based simulations using common random numbers
Random numbers are at the heart of every agent-based model (ABM) of health and disease. By representing each individual in a synthetic population, agent-based models enable detailed analysis of intervention impact and pa…
Evolutionary Planning in Latent Space
Planning is a powerful approach to reinforcement learning with several desirable properties. However, it requires a model of the world, which is not readily available in many real-life problems. In this paper, we propose…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Variance Reduction in Monte-Carlo Tree Search
Monte-Carlo Tree Search (MCTS) has proven to be a powerful, generic planning technique for decision-making in single-agent and adversarial environments. The stochastic nature of the Monte-Carlo simulations introduces err…
Decision MakingSimulation of Random LR Fuzzy Intervals
Random fuzzy variables join the modeling of the impreciseness (due to their ``fuzzy part'') and randomness. Statistical samples of such objects are widely used, and their direct, numerically effective generation is there…