UAV Control Optimization via Decentralized Markov Decision Processes
Unmanned aerial vehicle (UAV) swarm control has applications including target tracking, surveillance, terrain mapping, and precision agriculture. Decentralized control methods are particularly useful when the swarm is large, as centralized methods (a single command center controlling the UAVs) suffer from exponential computational complexity, i.e., the computing time to obtain the optimal control for the UAVs grow exponentially with the number of UAVs in the swarm in centralized approaches. Although many centralized control methods exist, literature lacks decentralized control frameworks with broad applicability. To address this knowledge gap, we present a novel decentralized UAV swarm control strategy using a decision-theoretic framework called decentralized Markov decision process (Dec-MDP). We build these control strategies in the context of two case studies: a) swarm formation control problem; b) swarm control for multitarget tracking. As most decision theoretic formulations suffer from the curse of dimensionality, we adapt an approximate dynamic programming method called nominal belief-state optimization (NBO) to solve the decentralized control problems approximately in both the case studies. In the formation control case study, the objective is to drive the swarm from a geographical region to another geographical region where the swarm must form a certain geometrical shape (e.g., selected location on the surface of a sphere). The motivation for studying such problems comes from data fusion applications with UAV swarms where the fusion performance depends on the strategic relative separation of the UAVs from each other. In the target tracking case study, the objective is the control the motion of the UAVs in a decentralized manner while maximizing the overall target tracking performance. Motivation for this case study comes from the surveillance applications using UAV swarms.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Decentralized Control of Partially Observable Markov Decision Processes using Belief Space Macro-actions
The focus of this paper is on solving multi-robot planning problems in continuous spaces with partial observability. Decentralized partially observable Markov decision processes (Dec-POMDPs) are general models for multi-…
Decision MakingCompositional Planning for Logically Constrained Multi-Agent Markov Decision Processes
Designing control policies for large, distributed systems is challenging, especially in the context of critical, temporal logic based specifications (e.g., safety) that must be met with high probability. Compositional me…
Decentralized Q-Learning for Stochastic Teams and Games
There are only a few learning algorithms applicable to stochastic dynamic teams and games which generalize Markov decision processes to decentralized stochastic control problems involving possibly self-interested decisio…
Q-LearningPlanning for Decentralized Control of Multiple Robots Under Uncertainty
We describe a probabilistic framework for synthesizing control policies for general multi-robot systems, given environment and sensor models and a cost function. Decentralized, partially observable Markov decision proces…
Scalable and Decentralized Algorithms for Anomaly Detection via Learning-Based Controlled Sensing
We address the problem of sequentially selecting and observing processes from a given set to find the anomalies among them. The decision-maker observes a subset of the processes at any given time instant and obtains a no…
Anomaly DetectionDecision Making