Safe Exploration in Linear Equality Constraint
With the extensive research and application, some shortcomings of reinforcement learning methods are gradually revealed. One of the considerable problems is that it is difficult for reinforcement learning methods to strictly satisfy the constraints. In this paper, a Singular Value Decomposition-based non-training method called 'Action Decomposition Regular' is proposed to achieve safe exploration. By adopting linear dynamics model, our method decomposes the action space into a constraint dimension and a free dimension for separate control, making policy strictly satisfy the linear equality constraint without limiting the exploration region. In addition, we show how our method should be used when the action space is limited and convex, which makes the method more suitable for real-world scenarios. Finally, we show the effectiveness of our method in a physically-based environment and prevail where reward shaping fails.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Safe ExplorationSimilar Papers 제목 키워드 기반
Enforcing Hard Linear Constraints in Deep Learning Models with Decision Rules
Deep learning models are increasingly deployed in safety-critical tasks where predictions must satisfy hard constraints, such as physical laws, fairness requirements, or safety limits. However, standard architectures lac…
FairnessENFORCE: Nonlinear Constrained Learning with Adaptive-depth Neural Projection
Ensuring neural networks adhere to domain-specific constraints is crucial for addressing safety and ethical concerns while also enhancing inference accuracy. Despite the nonlinear nature of most real-world tasks, existin…
HardNet++: Nonlinear Constraint Enforcement in Neural Networks
Enforcing constraint satisfaction in neural network outputs is critical for safety, reliability, and physical fidelity in many control and decision-making applications. While soft-constrained methods penalize constraint …
Integration of Prior Knowledge into Direct Learning for Safe Control of Linear Systems
This paper integrates prior knowledge into direct learning of safe controllers for linear uncertain systems under disturbances. To this end, we characterize the set of all closed-loop systems that can be explained by ava…
Safe Guaranteed Exploration for Non-linear Systems
Safely exploring environments with a-priori unknown constraints is a fundamental challenge that restricts the autonomy of robots. While safety is paramount, guarantees on sufficient exploration are also crucial for ensur…
Efficient ExplorationModel Predictive Control