Symbolic Opportunistic Policy Iteration for Factored-Action MDPs

2013-12-01 · NeurIPS 2013 12 · Aswin Raghavan, Roni Khardon, Alan Fern, Prasad Tadepalli

We address the scalability of symbolic planning under uncertainty with factored states and actions. Prior work has focused almost exclusively on factored states but not factored actions, and on value iteration (VI) compared to policy iteration (PI). Our ﬁrst contribution is a novel method for symbolic policy backups via the application of constraints, which is used to yield a new efﬁcient symbolic imple- mentation of modiﬁed PI (MPI) for factored action spaces. While this approach improves scalability in some cases, naive handling of policy constraints comes with its own scalability issues. This leads to our second and main contribution, symbolic Opportunistic Policy Iteration (OPI), which is a novel convergent al- gorithm lying between VI and MPI. The core idea is a symbolic procedure that applies policy constraints only when they reduce the space and time complexity of the update, and otherwise performs full Bellman backups, thus automatically adjusting the backup per state. We also give a memory bounded version of this algorithm allowing a space-time tradeoff. Empirical results show signiﬁcantly improved scalability over the state-of-the-art.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Reinforcement Learning for Control with Multiple Frequencies

2020-12-01 · NeurIPS 2020 12 · Jongmin Lee, ByungJun Lee, Kee-Eung Kim

Many real-world sequential decision problems involve multiple action variables whose control frequencies are different, such that actions take their effects at different periods. While these problems can be formulated wi…

continuous-control Continuous Control reinforcement-learning Reinforcement Learning+1

Neurosymbolic Reinforcement Learning with Formally Verified Exploration

2020-09-26 · NeurIPS 2020 12 · Greg Anderson, Abhinav Verma, Isil Dillig, Swarat Chaudhuri

We present Revel, a partially neural reinforcement learning (RL) framework for provably safe exploration in continuous state and action spaces. A key challenge for provably safe deep RL is that repeatedly verifying neura…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe Exploration

Learning a Policy for Opportunistic Active Learning

2018-08-29 · EMNLP 2018 10 · Aishwarya Padmakumar, Peter Stone, Raymond J. Mooney

Active learning identifies data points to label that are expected to be the most useful in improving a supervised model. Opportunistic active learning incorporates active learning into interactive tasks that constrain po…

Active Learning Object reinforcement-learning Reinforcement Learning+2

Formally Verified Approximate Policy Iteration

2024-06-11 · Maximilian Schäffeler, Mohammad Abdulaziz

We formally verify an algorithm for approximate policy iteration on Factored Markov Decision Processes using the interactive theorem prover Isabelle/HOL. Next, we show how the formalized algorithm can be refined to an ex…

FACMAC: Factored Multi-Agent Centralised Policy Gradients

2020-03-14 · NeurIPS 2021 12 · Bei Peng, Tabish Rashid, Christian A. Schroeder de Witt, Pierre-Alexandre Kamienny 외

We propose FACtored Multi-Agent Centralised policy gradients (FACMAC), a new method for cooperative multi-agent reinforcement learning in both discrete and continuous action spaces. Like MADDPG, a popular multi-agent act…

MuJoCo Multi-agent Reinforcement Learning Q-Learning Reinforcement Learning+3