A Model-free Biomimetics Algorithm for Deterministic Partially Observable Markov Decision Process
Partially Observable Markov Decision Process (POMDP) is a mathematical framework for modeling decision-making under uncertainty, where the agent's observations are incomplete and the underlying system dynamics are probabilistic. Solving the POMDP problem within the model-free paradigm is challenging for agents due to the inherent difficulty in accurately identifying and distinguishing between states and observations. We define such a difficult problem as a DETerministic Partially Observable Markov Decision Process (DET-POMDP) problem, which is a specific setting of POMDP. In this problem, states and observations are in a many-to-one relationship. The state is obscured, and its relationship is less apparent to the agent. This creates obstacles for the agent to infer the state through observations. To effectively address this problem, we convert DET-POMDP into a fully observable MDP using a model-free biomimetics algorithm called BIOMAP. BIOMAP is based on the MDP Graph Automaton framework to distinguish authentic environmental information from fraudulent data. Thus, it enhances the agent's ability to develop stable policies against DET-POMDP. The experimental results highlight the superior capabilities of BIOMAP in maintaining operational effectiveness and environmental reparability in the presence of environmental deceptions when compared with existing POMDP solvers. This research opens up new avenues for the deployment of reliable POMDP-based systems in fields that are particularly susceptible to DET-POMDP problems.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision Making Under UncertaintySimilar Papers 제목 키워드 기반
Learning in Observable POMDPs, without Computationally Intractable Oracles
Much of reinforcement learning theory is built on top of oracles that are computationally hard to implement. Specifically for learning near-optimal policies in Partially Observable Markov Decision Processes (POMDPs), exi…
Learning TheoryReinforcement Learning (RL)Learning Partially Observable Deterministic Action Models
We present exact algorithms for identifying deterministic-actions effects and preconditions in dynamic partially observable domains. They apply when one does not know the action model(the way actions affect the world) of…
Partially Observable Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)A Finite-State Controller Based Offline Solver for Deterministic POMDPs
Deterministic partially observable Markov decision processes (DetPOMDPs) often arise in planning problems where the agent is uncertain about its environmental state but can act and observe deterministically. In this pape…
Model-free Motion Planning of Autonomous Agents for Complex Tasks in Partially Observable Environments
Motion planning of autonomous agents in partially known environments with incomplete information is a challenging problem, particularly for complex tasks. This paper proposes a model-free reinforcement learning approach …
Motion PlanningQ-LearningIntegrating Algorithmic Planning and Deep Learning for Partially Observable Navigation
We propose to take a novel approach to robot system design where each building block of a larger system is represented as a differentiable program, i.e. a deep neural network. This representation allows for integrating a…
Deep LearningNavigateRobot NavigationState Estimation