Conformal Signal Temporal Logic for Robust Reinforcement Learning Control: A Case Study

2026-02-15 · Hani Beirami, M M Manjurul Islam arxiv

We investigate how formal temporal logic specifications can enhance the safety and robustness of reinforcement learning (RL) control in aerospace applications. Using the open source AeroBench F-16 simulation benchmark, we train a Proximal Policy Optimization (PPO) agent to regulate engine throttle and track commanded airspeed. The control objective is encoded as a Signal Temporal Logic (STL) requirement to maintain airspeed within a prescribed band during the final seconds of each maneuver. To enforce this specification at run time, we introduce a conformal STL shield that filters the RL agent's actions using online conformal prediction. We compare three settings: (i) PPO baseline, (ii) PPO with a classical rule-based STL shield, and (iii) PPO with the proposed conformal shield, under both nominal conditions and a severe stress scenario involving aerodynamic model mismatch, actuator rate limits, measurement noise, and mid-episode setpoint jumps. Experiments show that the conformal shield preserves STL satisfaction while maintaining near baseline performance and providing stronger robustness guarantees than the classical shield. These results demonstrate that combining formal specification monitoring with data driven RL control can substantially improve the reliability of autonomous flight control in challenging environments.

📄 PDF Abstract BibTeX arXiv:2602.14322

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Signal Temporal Logic Control Synthesis among Uncontrollable Dynamic Agents with Conformal Prediction

2023-12-07 · Xinyi Yu, Yiqi Zhao, Xiang Yin, Lars Lindemann

The control of dynamical systems under temporal logic specifications among uncontrollable dynamic agents is challenging due to the agents' a-priori unknown behavior. Existing works have considered the problem where eithe…

Conformal Prediction Prediction valid

Conformal Data-driven Control of Stochastic Multi-Agent Systems under Collaborative Signal Temporal Logic Specifications

2025-04-06 · Eleftherios E. Vlahakis, Lars Lindemann, Dimos V. Dimarogonas

We study the control of stochastic discrete-time linear multi-agent systems (MAS) subject to additive stochastic noise and collaborative signal temporal logic (STL) specifications to be satisfied with a desired probabili…

Conformal Prediction Model Predictive Control

Conformalized Signal Temporal Logic Inference under Covariate Shift

2026-03-28 · Yixuan Wang, Danyang Li, Matthew Cleaveland, Roberto Tron 외 arxiv

Signal Temporal Logic (STL) inference learns interpretable logical rules for temporal behaviors in dynamical systems. To ensure the correctness of learned STL formulas, recent approaches have incorporated conformal predi…

Deep reinforcement learning under signal temporal logic constraints using Lagrangian relaxation

2022-01-21 · Junya Ikemoto, Toshimitsu Ushio

Deep reinforcement learning (DRL) has attracted much attention as an approach to solve optimal control problems without mathematical models of systems. On the other hand, in general, constraints may be imposed on optimal…

Decision Making Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Funnel-based Reward Shaping for Signal Temporal Logic Tasks in Reinforcement Learning

2022-11-30 · Naman Saxena, Gorantla Sandeep, Pushpak Jagtap

Signal Temporal Logic (STL) is a powerful framework for describing the complex temporal and logical behaviour of the dynamical system. Numerous studies have attempted to employ reinforcement learning to learn a controlle…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)