Algorithms for Learning Markov Field Policies

2012-12-01 · NeurIPS 2012 12 · Abdeslam Boularias, Jan R. Peters, Oliver B. Kroemer

We present a new graph-based approach for incorporating domain knowledge in reinforcement learning applications. The domain knowledge is given as a weighted graph, or a kernel matrix, that loosely indicates which states should have similar optimal actions. We first introduce a bias into the policy search process by deriving a distribution on policies such that policies that disagree with the provided graph have low probabilities. This distribution corresponds to a Markov Random Field. We then present a reinforcement and an apprenticeship learning algorithms for finding such policy distributions. We also illustrate the advantage of the proposed approach on three problems: swing-up cart-balancing with nonuniform and smooth frictions, gridworlds, and teaching a robot to grasp new objects.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Model-Free Mean-Field Reinforcement Learning: Mean-Field MDP and Mean-Field Q-Learning

2019-10-28 · René Carmona, Mathieu Laurière, Zongjun Tan

We study infinite horizon discounted Mean Field Control (MFC) problems with common noise through the lens of Mean Field Markov Decision Processes (MFMDP). We allow the agents to use actions that are randomized not only a…

General Reinforcement Learning Q-Learning Reinforcement Learning Reinforcement Learning (RL)

Non-Markovian policies occupancy measures

2022-05-27 · Romain Laroche, Remi Tachet des Combes, Jacob Buckman

A central object of study in Reinforcement Learning (RL) is the Markovian policy, in which an agent's actions are chosen from a memoryless probability distribution, conditioned only on its current state. The family of Ma…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

The Complexity of Markov Equilibrium in Stochastic Games

2022-04-08 · Constantinos Daskalakis, Noah Golowich, Kaiqing Zhang

We show that computing approximate stationary Markov coarse correlated equilibria (CCE) in general-sum stochastic games is computationally intractable, even when there are two players, the game is turn-based, the discoun…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Periodic agent-state based Q-learning for POMDPs

2024-07-08 · Amit Sinha, Matthieu Geist, Aditya Mahajan

The standard approach for Partially Observable Markov Decision Processes (POMDPs) is to convert them to a fully observed belief-state MDP. However, the belief state depends on the system model and is therefore not viable…

Q-Learning Reinforcement Learning (RL)

Safe Policy Improvement Approaches on Discrete Markov Decision Processes

2022-01-28 · Philipp Scholl, Felix Dietrich, Clemens Otte, Steffen Udluft

Safe Policy Improvement (SPI) aims at provable guarantees that a learned policy is at least approximately as good as a given baseline policy. Building on SPI with Soft Baseline Bootstrapping (Soft-SPIBB) by Nadjahi et al…