Sequential learning and control: Targeted exploration for robust performance
We present a novel dual control strategy for uncertain linear systems based on targeted harmonic exploration and gain-scheduling with performance and excitation guarantees. In the proposed sequential approach, robust control is implemented after exploration with the main feature that the exploration is optimized with respect to the robust control performance. Specifically, we leverage recent results on finite excitation using spectral lines to determine a high probability lower bound on the resultant finite excitation of the exploration data. This provides an a priori upper bound on the remaining model uncertainty after exploration, which can further be leveraged in a gain-scheduling controller design that guarantees robust performance. This leads to a semidefinite program-based design which computes an exploration strategy with finite excitation bounds and minimal energy, and a gain-scheduled controller with probabilistic performance bounds that can be implemented after exploration. The effectiveness of our approach and its benefits over common random exploration strategies are demonstrated with an example of a system which is 'hard to learn'.
Code (0)
등록된 구현이 없습니다.
Tasks
SchedulingSimilar Papers 제목 키워드 기반
Targeted Exploration via Unified Entropy Control for Reinforcement Learning
Recent advances in reinforcement learning (RL) have improved the reasoning capabilities of large language models (LLMs) and vision-language models (VLMs). However, the widely used Group Relative Policy Optimization (GRPO…
Reinforcement LearningSPARC-RAG: Adaptive Sequential-Parallel Scaling with Context Management for Retrieval-Augmented Generation
Retrieval-Augmented Generation (RAG) grounds large language model outputs in external evidence, but remains challenged on multi-hop question answering that requires long reasoning. Recent works scale RAG at inference tim…
Multi-hop Question AnsweringCoRefine: Confidence-Guided Self-Refinement for Adaptive Test-Time Compute
Large Language Models (LLMs) often rely on test-time scaling via parallel decoding (for example, 512 samples) to boost reasoning accuracy, but this incurs substantial compute. We introduce CoRefine, a confidence-guided s…
Targeted Deep Learning System Boundary Testing
Evaluating the behavioral boundaries of deep learning (DL) systems is crucial for understanding their reliability across diverse, unseen inputs. Existing solutions fall short as they rely on untargeted random, model- or …
Deep LearningDNN Testingimage-classificationImage ClassificationAn Optimal Policy for Learning Controllable Dynamics by Exploration
Controllable Markov chains describe the dynamics of sequential decision making tasks and are the central component in optimal control and reinforcement learning. In this work, we give the general form of an optimal polic…
Reinforcement LearningDecision Making