Symbolic Network: Generalized Neural Policies for Relational MDPs

2020-02-18 · Sankalp Garg, Aniket Bajpai, Mausam

A Relational Markov Decision Process (RMDP) is a first-order representation to express all instances of a single probabilistic planning domain with possibly unbounded number of objects. Early work in RMDPs outputs generalized (instance-independent) first-order policies or value functions as a means to solve all instances of a domain at once. Unfortunately, this line of work met with limited success due to inherent limitations of the representation space used in such policies or value functions. Can neural models provide the missing link by easily representing more complex generalized policies, thus making them effective on all instances of a given domain? We present SymNet, the first neural approach for solving RMDPs that are expressed in the probabilistic planning language of RDDL. SymNet trains a set of shared parameters for an RDDL domain using training instances from that domain. For each instance, SymNet first converts it to an instance graph and then uses relational neural models to compute node embeddings. It then scores each ground action as a function over the first-order action symbols and node embeddings related to the action. Given a new test instance from the same domain, SymNet architecture with pre-trained parameters scores each ground action and chooses the best action. This can be accomplished in a single forward pass without any retraining on the test instance, thus implicitly representing a neural generalized policy for the whole domain. Our experiments on nine RDDL domains from IPPC demonstrate that SymNet policies are significantly better than random and sometimes even more effective than training a state-of-the-art deep reactive policy from scratch.

📄 PDF Abstract BibTeX arXiv:2002.07375

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Test 설명 없음

Generalized Neural Policies for Relational MDPs

2020-01-01 · ICML 2020 1 · Sankalp Garg, Aniket Bajpai, Mausam

Solving Relational MDPs with Exogenous Events and Additive Rewards

2013-06-26 · S. Joshi, R. Khardon, P. Tadepalli, A. Raghavan 외

We formalize a simple but natural subclass of service domains for relational planning problems with object-centered, independent exogenous events and additive rewards capturing, for example, problems in inventory control…

Deep Explainable Relational Reinforcement Learning: A Neuro-Symbolic Approach

2023-04-17 · Rishi Hazra, Luc De Raedt

Despite numerous successes in Deep Reinforcement Learning (DRL), the learned policies are not interpretable. Moreover, since DRL does not exploit symbolic relational representations, it has difficulties in coping with st…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning

Multi-Environment POMDPs: Discrete Model Uncertainty Under Partial Observability

2025-10-27 · Eline M. Bovy, Caleb Probine, Marnix Suilen, Ufuk Topcu 외 arxiv

Multi-environment POMDPs (ME-POMDPs) extend standard POMDPs with discrete model uncertainty. ME-POMDPs represent a finite set of POMDPs that share the same state, action, and observation spaces, but may arbitrarily vary …

Symbolic Dynamic Programming for Continuous State and Observation POMDPs

2012-12-01 · NeurIPS 2012 12 · Zahra Zamani, Scott Sanner, Pascal Poupart, Kristian Kersting

Partially-observable Markov decision processes (POMDPs) provide a powerful model for real-world sequential decision-making problems. In recent years, point- based value iteration methods have proven to be extremely effec…

Decision Making Sequential Decision Making