Regret Minimization in Partially Observable Linear Quadratic Control
We study the problem of regret minimization in partially observable linear quadratic control systems when the model dynamics are unknown a priori. We propose ExpCommit, an explore-then-commit algorithm that learns the model Markov parameters and then follows the principle of optimism in the face of uncertainty to design a controller. We propose a novel way to decompose the regret and provide an end-to-end sublinear regret upper bound for partially observable linear quadratic control. Finally, we provide stability guarantees and establish a regret upper bound of $\tilde{\mathcal{O}}(T^{2/3})$ for ExpCommit, where $T$ is the time horizon of the problem.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting
We study the problem of adaptive control in partially observable linear quadratic Gaussian control systems, where the model dynamics are unknown a priori. We propose LqgOpt, a novel reinforcement learning algorithm based…
Reinforcement LearningLogarithmic Regret Bound in Partially Observable Linear Dynamical Systems
We study the problem of system identification and adaptive control in partially observable linear dynamical systems. Adaptive and closed-loop system identification is a challenging problem due to correlations introduced …
counterfactualThe Partially Observable History Process
We introduce the partially observable history process (POHP) formalism for reinforcement learning. POHP centers around the actions and observations of a single agent and abstracts away the presence of other players witho…
Formreinforcement-learningReinforcement LearningReinforcement Learning (RL)HSVI can solve zero-sum Partially Observable Stochastic Games
State-of-the-art methods for solving 2-player zero-sum imperfect information games rely on linear programming or regret minimization, though not on dynamic programming (DP) or heuristic search (HS), while the latter are …
Decision MakingHeuristic SearchOpen-Ended Question AnsweringSequential Decision MakingRegret Analysis of Learning-Based Linear Quadratic Gaussian Control with Additive Exploration
In this paper, we analyze the regret incurred by a computationally efficient exploration strategy, known as naive exploration, for controlling unknown partially observable systems within the Linear Quadratic Gaussian (LQ…
Efficient Exploration