Optimized Monte Carlo Tree Search for Enhanced Decision Making in the FrozenLake Environment

2024-09-25 · Esteban Aldana Guerra

Monte Carlo Tree Search (MCTS) is a powerful algorithm for solving complex decision-making problems. This paper presents an optimized MCTS implementation applied to the FrozenLake environment, a classic reinforcement learning task characterized by stochastic transitions. The optimization leverages cumulative reward and visit count tables along with the Upper Confidence Bound for Trees (UCT) formula, resulting in efficient learning in a slippery grid world. We benchmark our implementation against other decision-making algorithms, including MCTS with Policy and Q-Learning, and perform a detailed comparison of their performance. The results demonstrate that our optimized approach effectively maximizes rewards and success rates while minimizing convergence time, outperforming baseline methods, especially in environments with inherent randomness.

📄 PDF Abstract BibTeX arXiv:2409.16620

Code (0)

등록된 구현이 없습니다.

Tasks

Decision MakingQ-Learning

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Monte Carlo Search Algorithms Discovering Monte Carlo Tree Search Exploration Terms

2024-04-14 · Tristan Cazenave

Monte Carlo Tree Search and Monte Carlo Search have good results for many combinatorial problems. In this paper we propose to use Monte Carlo Search to design mathematical expressions that are used as exploration terms f…

Bayesian Optimized Monte Carlo Planning

2020-10-07 · John Mern, Anil Yildiz, Zachary Sunberg, Tapan Mukerji 외

Online solvers for partially observable Markov decision processes have difficulty scaling to problems with large action spaces. Monte Carlo tree search with progressive widening attempts to improve scaling by sampling fr…

Bayesian Optimization

Accelerating Monte-Carlo Tree Search with Optimized Posterior Policies

2026-01-03 · Keith Frankston, Benjamin Howard arxiv

We introduce a recursive AlphaZero-style Monte--Carlo tree search algorithm, "RMCTS". The advantage of RMCTS over AlphaZero's MCTS-UCB is speed. In RMCTS, the search tree is explored in a breadth-first manner, so that ne…

Single-Agent Optimization Through Policy Iteration Using Monte-Carlo Tree Search

2020-05-22 · Arta Seify, Michael Buro

The combination of Monte-Carlo Tree Search (MCTS) and deep reinforcement learning is state-of-the-art in two-player perfect-information games. In this paper, we describe a search algorithm that uses a variant of MCTS whi…

Deep Reinforcement Learning

Hedging of Financial Derivative Contracts via Monte Carlo Tree Search

2021-02-11 · Oleg Szehr

The construction of approximate replication strategies for pricing and hedging of derivative contracts in incomplete markets is a key problem of financial engineering. Recently Reinforcement Learning algorithms for hedgi…

Q-Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)